Developing the next generation of 3D game interface (multiple teams possible)

The goal of this project is to employ a recent generation time-of-flight depth camera as input device for a game. As opposed to normal video cameras, a depth camera delivers 3D information rather than intensity information in images. This new technology has only recently reached a high level of maturity, robustness and data quality. Latest models deliver high-quality depth data at up to 60 frames per second. Our lab has several of these cameras available to experiment with. One of the cameras was explicitly lend to us by the manufacturer with the goal of having bright students work on great game demos.

In this project project, you will therefore explore the suitability of the sensor as a real-time input device for games. Seeing at what rich form of data the camera delivers, it is quite clear that it will eventually revolutionize the way how games are played. Recent new interface technologies like the Wii only provide a fraction of the capabilities that a sensor like ours provides. In this project, you will have the chance to play a significant role in that development as long as the technology is still fresh out of the lab.

The project has several components:

• Use your imagination and develop a cool game scenario. Ideas could be Dance Revolution 2 where you control a dancer by moving in front of the camera, a boxing or martial arts game where you do real kicks, a flight or car simulator where you control the game by gestures etc.
• Develop a tracking method for the depth camera to get all the input information you need in real-time. This is quite an interesting problem from a scientific point of view and has definite paper potential.
• Implement a game prototype, have fun with it and show it to us

In fact we can easily split this project into many sub-projects, so each team of two or three people could work on a separate game idea. However, I guess I will at most be able to supervise two teams. So, please decide soon if you want to join this very fun project.

• It will be huge fun to play with cool cutting edge sensors.
• You have the possibility to really make a contribution in the field of computer vision by developing robust tracking
• It will be huge fun because it requires creativity on many levels, ranging from the design of the game idea to the design of the tracking algorithm
• There is a big chance that one of the manufacturers of the depth cameras who lent them to us explicitly for the class will be very interested in your developments - but tdont' worry, there is no commitment to the manufacturer of any form if you do this project; it is just another opportunity

Existing Infrastructure

References

• Thrun et al., Probabilistic Robotics, MIT Press, Chapters 2-4 on filtering and tracking
• An early paper on fast body tracking: PDF
• A classical paper on particle filter based tracking: (PS.gz)
4) a more recent paper on tracking with depth cameras: PDF

• http://www.canesta.com/assets/pdf/technicalpapers/CVPR_Submission_TOF.pdf
• http://www.mesa-imaging.ch/publications.php

Contact