Development of Neuro-Stereo-Display for 3D Visualization
Tech Area / Field
- INF-ELE/Microelectronics and Optoelectronics/Information and Communications
3 Approved without Funding
FIAN Lebedev, Russia, Moscow
- VNIITF, Russia, Chelyabinsk reg., Snezhinsk
- Brigham Young University, USA, UT, Provo
Project summaryThe aim of the proposed project is the development of stereo 3D visualization system, which requires neither special glasses nor display frequency doubling. The proposed system is based on fast neural network image processing. The latter includes both video compression and stereo frame synthesis. These problems are fundamental for 3D vision with its extremely high information flows. Neurocomputing proved to be a proper tool in solving such kind of ill-posed problems.
In a simple setting the proposed scheme comprises a pair of semi-transparent LCD's and makes use of the capability of neural hardware to reconstruct appropriate frames for these LCD's in real time. The above scheme is mainly illustrative. The basic idea is to exploit the power of neural computation for intermediate view reconstruction and stereo image video compression.
The proposed project is the advanced stage of the ISTC project #031-94 (see WWW site http://canopus.lpi.msk.su/project31). The latter resulted in development of up to date neural computing system Nimfa with sophisticated client-server software and neural acceleration board. Nimfa (Neural Information Methods For Applications) is designed as a tool for development neural network applications. Many useful applications may benefit from the power of massively parallel neurocomputing, provided by relatively cheap neural acceleration boards. Stereo vision is one of prospective areas for such PC-based boards. They may significantly change the way we interact with computer, paving the way for a real 3D virtual reality interface.
To date stereo vision applications include:
- Real time high fidelity training simulators and virtual reality systems
- Industrial design and scientific visualization (multidimensional simulations results, complicated engineering drawing in CAD, image viewing in architecture)
- Monitoring (stereo vision in robotics; monitoring of complex technical systems, e.g. 3D radiation and thermal fields in nuclear reactors, comprehensive video support for aviation dispatchers, non-destructive evaluation of engineered constructions)
- Medicine (tomographic imaging in medicine, viewing of complex molecules in pharmacology, gene engineering)
- 3D-Windows computer interface.
All these applications require real time image processing. Existing specialized image processors are successfully used for most typical visual transformations (e.g. real-time projections of rotating 3D objects, contrasting, etc.). Stereo vision raises the brand-new class of problems related to multi-view image reconstruction. This project proposes a unified concept of multi-view images compression and reconstruction with neural networks. Neurocomputing combines massive parallelism with relatively simple programming and flexible self-adaptation.
The major expected results of the project include:
- Stereo display prototype generating stereo pairs and sequences of stereo images.
- Conceptual scheme of multi-view display (for realistic 3D vision) with the prototypes of main components.
- Neural network algorithms for image processing, including data compression, reconstruction of intermediate views, contrasting, etc.
- Appropriate neural hardware for stereo displays.
The team of the proposed project is currently running the ISTC project #31-94. This team comprises 24 former nuclear weapon specialists who now became professional neuro-computer scientists due to the ISTC support. The Laboratory for Artificial Neural Networks was founded in VNIITF in 1996. Thus the suggested project directly matches the ISTC purposes.
Potential Role of Foreign Collaborators
Some optical schemes of stereo display may require unique technologies. These specimens might be ordered from foreign collaborators. But the real concern is pushing on to the market technical solutions elaborated within the proposed project. Specialists from the firms presenting in the market could specify the industrial needs and help to judge between technical alternatives. This will undoubtedly help the project to match its goals.
The International Science and Technology Center (ISTC) is an intergovernmental organization connecting scientists from Kazakhstan, Armenia, Tajikistan, Kyrgyzstan, and Georgia with their peers and research organizations in the EU, Japan, Republic of Korea, Norway and the United States.
ISTC facilitates international science projects and assists the global scientific and business community to source and engage with CIS and Georgian institutes that develop or possess an excellence of scientific know-how.