The 102nd Installment
Video Games and VR

by Hiroaki Tobita,
Professor, Master Program of Information Systems Architecture

With ground-breaking improvements to the performance of graphics cards and screens comes greater possibilities for the enjoyment of video games. Graphics cards now allow for more complex and realistic visual effects to run smoothly on screen while large 4K monitors and head-mounted displays (HMDs) provide users with the feeling of immersion in game spaces. There is even a movement to sell older games and consoles in bundle packs. These older games include a number of masterpieces which pursued the best entertainment under limiting restrictions.

There are a large number of approaches to video game augmentation which this article aims to introduce. In particular, screen augmentation in augmented reality (AR) and virtual reality (VR). AR systems can turn desks and walls into interactive surfaces, while VR systems create an immersive feeling by turning an entire room into a virtual space. Conversations that look at how users that utilize systems (including visualizations) interact with information is important. AR and VR are also being researched as tools for video game augmentation. A part of this is the way these tools use a variety of sensors (primarily cameras) to realize these forms of artistic media.

IllumiRoom from Microsoft Research is a video game augmentation system which projects visualizations around a television screen to create an expansive visual effect. This system uses stereoscopic cameras to detect what is on the screen and the environment around it and then the projector projects an extension of the footage around the screen, providing a compelling visual effect to the user. However, due to the fixed locations of the camera and projector, the room mapping and projection features are limited to set directions. RoomAlive, which uses IllumiRoom to augment the entirety of a room, can project visualizations onto all walls in a room, using multiple projectors and stereoscopic cameras installed to the ceiling. Approaches such as IllumiRoom and RoomAlive, which require the installation of large projectors onto ceilings, are not easy to install and utilize in one’s home living room. However, with projectors becoming smaller and cameras more advanced, it is not hard to think that research into systems such as these will become an everyday occurrence in the near future.

Another approach is an AR system which utilizes a battery-powered portable projector. As the entire system is small and light, it is possible to use while carrying in one’s hand. Furthermore, it is capable of understanding AR markers and projection areas to a greater extent by combining projectors and cameras. A wide variety of games are possible by combining our expansive understandings of technology while operating projection systems like a flashlight. However, while one can implement simple systems with the use of small projectors, they are met with limitations that are not present in existing AR systems. On the other hand, with the existence of projectors integrated into mobile devices (including smartphones), there is a hidden potential for new forms of gameplay and communication.

In 2018, Japan had a large number of exhibitions at an international AR and VR conferences. The ACM Symposium on Virtual Reality Software and Technology (VRST) is, as the name suggests, a conference that focuses on VR and AR technologies, where a variety of talks were presented including those covering ACM Interactive Surfaces and Spaces (ISS) and VR software and hardware. Furthermore, at the ACM SIGGRAPH Asia conference (the Asian branch of the SIGGRAPH Conferences), we can learn about the latest technology in computer graphics. In addition to spoken lectures, attendees can have hands-on experiences with systems through demonstrations. The SIGGRAPH’s Emerging Technology session in particular is a great opportunity to learn about cutting-edge technology through demonstrations.