Sistema Híbrido Baseado em Geometria e Vídeo 360º 3D para Realidade Virtual em Dispositivos Móveis

Abstract

Nowadays virtual reality is becoming popular in society because it provides an immersive experience at lower costs and better usability that it had in the past. However, it turns out that the development of the devices used to spread this technology tends to diverge in two different ways. On the one hand, there are devices that use the computing power of fixed computers allowing a high-quality experience but that limit the user's ability to move due to the necessary cables. On the other hand, there is the growth of mobile devices that allow greater freedom of movement but limit the experience due to the fact that they have less processing capacity. To promote virtual reality in mobile devices with high quality and visual fidelity without frame rate losses and without prejudice in the immersive experience, it is necessary to simplify the virtual environments visualized, thus reducing the computational requirements. A possible compromise is the combination of objects with three-dimensional geometry superimposed on a 360º image or video that covers the user's viewing area. This 360º image or video is previously generated on a high-processing power system, thus containing the details needed to provide good visual quality. The generated 360º image or video cannot provide the interaction that is typically needed in virtual reality environments, but overlapping real-time geometry may allow for interactivity with specific objects of the scenario. However, this solution causes coherence problems between 3D objects and the pre-generated background, such as collisions, occlusions, lighting, shadows, and transparency maintenance along with problems with data streaming and background video compression. Therefore, a set of well-defined metadata must be preserved from the scene that originated the background video, and used in the final device with appropriate techniques to tackle the problems of coherence. In this work we propose a technology aimed at creating a hybrid system based on 3D objects superimposed on the 360º image or video to provide virtual reality environments in devices with low processing capacity. The proposed solution was duly tested and validated through quantitative and qualitative measures. It is expected that such an approach will enable higher quality on mobile-based experiences, which in turn will have an impact on the cost of systems and the progression of the new technology in a larger part of society.