HOLONICS Project IST-2004-511318
(2000-2003).
Keywords: EC VI Framework. IST. Holographic Displays. Computer
Vision. Real-Time. Motion Capture. Human Body 3D digitizing.
Project Summary
Leisure and entertainment require every day more innovation; innovation that must be understood in terms of new contents but also in terms of new user interaction paradigms. 3D is at the forefront of community research efforts today. HOLONICS has as its main objective to research new technologies for the automatic acquisition, management, presentation and delivery of 3D information.
HOLONICS will place holographic technology and computer vision based human action capture at the base of its research efforts with the objective to achieve most realistic representations of 3D contents for multiple simultaneous users, real-time automatic acquisition of human action data (geometry, volume, texture) and unconstrained user interaction with the system and / or other users. HOLONICS will have as one of its main requirements to avoid the use of any kind of device that can be annoying, limiting or little comfortable for the user (i.e. head mounted devices, markers, cabling, etc); a circumstance that have jeopardised the development of the sector in the past.
HOLONICS’s demonstration applications will explore the possibilities of holographic display technology for true 3D representation of contents, will feature new user interaction capabilities using the human body and will develop new methods for the automatic acquisition of 3D human body information. HOLONICS Tele-presence application will use a set-up of cameras to acquire in real time the volumetric structure and the texture of the full body of a person. Then, the system will track and animate the resulting 3D model with the objective to transmit the information to a remote 3D holographic display monitor for naked eye interpersonal communications. The Virtual Theatre application will use large panels of holographic displays combined with sophisticated human action capture to demonstrate HOLONICS’s can be successfully applied to wide scenarios and highly interactive experiences that can be shared by a great number of users.
Project Objectives
HOLONICS is born with the objective to become the reference project in the field of acquisition, presentation and management of 3D information for the leisure and entertainment sectors. HOLONICS intends to analyse the complete 3D development chain; a process that includes capturing, displaying, manipulating and interacting with 3D information. A concept supported on highly immesive scenarios where the users can share their experiences with other users and are no longer constrained by: annoying cabling, primitive markers, uncomfortable wearable devices, special lighting, limited freedom to move, etc. Those objectives must be achieved under the additional constrains of real-time performance and careful attention to usability and feasibility issues.
HOLONICS intends to realise the above requirements by achieving two core scientific objectives:
-
It is necessary to build a computer vision 3D-reconstruction system based on video image streams.
-
The 3D reconstruction will be based on self-calibration techniques. No calibration apparatus will be allowed.
Actually, it is possible to display VRML models in commercial Internet browsers, last generation set-top boxes and information desks. Of course VRML is also suitable for CD-ROMS and most popular 3D modelling packages.
Project Outcomes
The project is expected to provide the following results:
-
Human Action Capture. Computer vision techniques aimed at capturing human action in real time: motion, volume / geometry and texture. Using a set-up of cameras arranged in the scenario, the system will acquire action information from the user while avoiding the use of markers or uncomfortable wearable devices. The system must be able to acquire photo realistic 3D motion data as well as volumetric information and texture of the body of the user in close to real time.
-
Holographic displays. Holographic techniques offer the possibility of constructing highly immersive 3D experiences in front of the user without being bothered by uncomfortable wearable devices (i.e. helmets or positioning sensors). A 3D experience where objects may appear behind or in front of the display and can be shared by several users simultaneously. Two variants will be developed:
- 3D monitor holographic display (aimed at personal use or reduced number of simultaneous users). The display will have a FOV ~50, screen diagonal 30", objects floating in the "empty" display frame, 80-90 cm in front of the last large-size imaging element.
- Large-scale Panel holographic display (aimed at presentations in wide scenarios). The display will be a 100Mpixel system, with approx. 1,6m x 3,2m holographic screen surface, composed of ~80 cm wide panels.
In addition, to the two core objectives described above, the project proposes to explore as a secondary objective the combination of both technologies with the aim to make possible the new paradigm for advanced acquisition, delivery and presentation of 3D multimedia contents. For that purpose, two applications demonstrators will be developed who will integrate action capture and holographic displays into a single production chain: the tele-presence and virtual theatre applications (see descriptions in section 7.1.3.5 ). While the first application is conceived as a self-contained tool for a very concrete use, the virtual theatre application is designed as a test bed where to exercise advanced HOLONICS features.
In that sense, our scale of ambition proposes a successful HOLONICs deployment would consist of an action capture system capable of acquiring photo-realistic motion and geometry data of a human body in close to real time. The data would be displayed in real time on a monitor and / or panel type holographic display where the proposed demonstrators will be exercised.
