Free Viewpoint Video (FVV) might become one of the most interesting features in future digital television, enabling viewers to move around freely and observe the scene in three dimensions (3D) from any desired viewpoint, similar to many 3D interactive computer graphics applications today. This paper describes a FVV appropriate for virtual reality systems that allows the user to explore a 3D scene in real time using a small set of sparse cameras. Our method combines a simple geometric model of the scene composed of planes with tracking information of moving targets to generate novel perspective corrected 3D views of a virtual camera. To achieve real-time rendering frame rates, view dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. Our current real-time prototype is able to generate an interactive 2D virtual camera view from 3 synchronized video streams of resolution 768×576 with several moving objects at about 12 fps.
BibTeX
@inproceedings{Silva:SVR2010,
author = {Jeferson R. da Silva and Thiago T. Santos and Carlos H. Morimoto},
title = {Real Time Novel View Scene Rendering From Multiple Sparse Videos},
booktitle = {Proceedings of the XII Brazilian Symposium on Virtual and Augmented Reality},
series = {SVR '10},
year = {2010},
publisher = {Sociedade Brasileira de Computação},
address = {Natal, Brazil},
pages = {184-193}
}