A player using the Oculus Rift virtual reality headset and associated controllers to control a game
A player using the Oculus Rift virtual reality headset and associated controllers to control a game

A virtual reality game or VR game is a video game played on virtual reality (VR) hardware. Most VR games are based on player immersion, typically through head-mounted display unit or headset and one or more controllers. The headset typically provides two stereoscopic displays in front of the user's eyes to simulate a 3D space.

The video game industry made early attempts at VR in the 1980s, most notably with Mattel's Power Glove and Nintendo's Virtual Boy. With the introduction of the first consumer-ready VR product, the Oculus Rift, in 2013, VR games soon followed, including existing games adapted for the VR hardware, and new games designed directly for VR. While VR hardware and games grew modestly for the remainder of the 2010s, Half-Life: Alyx, a full VR game developed by Valve, was considered the killer application for VR games.

History

See also: Virtual reality § History

Early VR games (1980s-2000s)

Research into virtual reality (VR) hardware and software started as early as 1968 by Ivan Sutherland and his student Bob Sproull , but most equipment was too expensive for consumer use, and its use for games was limited. The first VR head mounted display that was connected to a computer. In the late 1980s, Jaron Lanier and Thomas G. Zimmerman, former programmers for Atari, Inc., began developing hardware under the name VPL Research, with Lanier coining the term "virtual reality" for their products.[1] One of VPL's products was the VPL DataGlove, a glove that sensed the user's finger movement and translated it into computer input. The idea inspired engineers at Abrams/Gentile Entertainment (AGE) to work with Mattel and Nintendo to build a low-cost version of the DataGlove to work with the Nintendo Entertainment System (NES), omitting much of the technical sophistication and movement sensitivity of the DataGlove as to achieve a reasonable consumer cost.[2] The Power Glove was released in 1989.[3] The games Super Glove Ball and Bad Street Brawler were specifically designed to use the Power Glove, while other NES games could be played using the Power Glove by mapping its output to various controls. About one million Power Glove units were sold before Mattel discontinued it in 1990. Its low cost compared to the DataGlove and other similar gloves led academics to buy the unit for their own research.[2]

The heads-up viewscreen of Nintendo's Virtual Boy
The heads-up viewscreen of Nintendo's Virtual Boy

Interest in VR grew in the 1990s, particularly after the 1992 film Lawnmower Man, which helped popularize the idea of VR headsets with the general public.[1] Reflection Technology, Inc. (RTI) had been developing a head-mounted, stereoscopic head-tracking system using light-emitting diode (LED) displays, the Private Eye. One application they had tested included a tank game. Seeking funding for larger production, RTI licensed the technology to Nintendo, and under Gunpei Yokoi, Nintendo developed the Virtual Boy, released in 1995.[4] The Virtual Boy used red LED displays rather than full-color ones, as they were the most inexpensive to produce, and required mounting to a stand to be played, rather than head-mounted. The system was thus awkward to use both from looking into the viewer and the eyestrain from the red LEDs. Only 22 games were produced for the Virtual Boy, and it was considered to be one of Nintendo's commercial failures.[4] Sega developed a low-cost VR device, the Sega VR, in the early 1990s, for its arcade games and home consoles; the unit did not advance beyond the prototype stage, though Sega incorporated some of its head-tracking technology into its arcade cabinets.[5]

VR systems without head-mounted hardware were also developed in the 1990s, including the Cave automatic virtual environment (CAVE). CAVE systems included multiple flat screen displays, typically at least three walls to surround the human player, and incorporated some type of tracking sensor system to match the images on the walls to what direction the player was looking. Early applications of CAVE system were game-based demonstrations, but the cost remained prohibitive for commercial deployment even through the 2010s.[6][7]

Around this same time in the 1990s, major innovations in real-time 3D graphics had been made across computer, console, and arcade games, and with further improvements in affordable consumer technologies, arcade games began to decline as they could not compete with these innovations. Arcade game manufacturers instead focused on offering games that could not easily be replicated at home, which included the introduction of VR-based arcade games. For example, the Virtuality Group produced its Virtuality line of arcade games starting in the early 1990s that typically included a VR headset with head-tracking and other features. However, the cost and upkeep of these machines made it difficult to continue support for them.[8]

There remained strong interest from academics to explore what VR, along with augmented reality and other mixed reality systems, could bring to video games, through the 2000s, but these games were mostly prepared for research proof-of-concepts to demonstrate the interaction of VR hardware, software, and human motion rather than for commercial release, since hardware costs were still high.[9]

Introduction of consumer-ready hardware (2010s–present)

After decades of attempts from its introduction, low-cost, consumer-grade VR hardware began to appear in the 2010s.[7] The Oculus Rift is considered the first consumer-ready VR headset and was first released in 2016. The unit was developed by Palmer Luckey, and first announced in 2013 as an inexpensive VR option for video games.[10] During testing, Luckey had gained the help of id Software's John Carmack to develop a VR version of Doom 3 for Oculus. While this helped to successfully demonstrate the Oculus, which led to Facebook acquiring Oculus in 2014 for $2 billion,[11] it also led to a lawsuit between ZeniMax Media, id's parent company, against Oculus over intellectual property theft over Carmack's participation. The case was settled out of court.[12] Nine games were available at launch and Oculus had established a number of partnerships to provide more games following its release.[13][14]

The Oculus Rift announcement led to other VR developments. Sony Computer Entertainment developed the PlayStation VR system for the PlayStation 4, while Valve partnered with HTC to develop the HTC Vive; both of these units were released in 2016.[15][16] Valve later developed their own headset without HTC, the Valve Index, which was released in 2019.[17]

A Google Cardboard headset
A Google Cardboard headset

Headsets designed to accommodate mobile devices were also released, using the devices to create the stereoscopic display, some of the positioning functions, and other VR tracking embedded in the additional hardware. In 2014, Google introduced Google Cardboard, a inexpensive headset package that constructed from cardboard for use with Android phones; the completed headset creates the necessary visual space to support stereoscopic view from the phone's display.[18] Samsung, in conjunction with Oculus, released the Samsung Gear VR in 2015 to support its Samsung Galaxy smartphones; services for the Gear VR ended in 2020.[19] Nintendo released the Nintendo Labo VR Kit in 2019 as part of its Labo series of toys-to-life cardboard products.[20] A handful of Nintendo Switch games support Labo VR functionality, such as the 2017 games Super Mario Odyssey and The Legend of Zelda: Breath of the Wild.[21]

Despite the availability of low-cost hardware for VR, the technology had still not taken off for video games by 2018 as had been expected when the Oculus Rift was announced. This was attributed to the lack of a killer application, a game that would drive people to buy the hardware to play it. There had been several games from smaller studios that had been considered successful, such as Superhot VR and Beat Saber, but the triple-A studios had not ventured into the area. Sales of VR hardware had been steadily increasing since 2016, but were still under 10 million units by 2018, and there were signs manufacturers were starting to back off in this area.[22][23][24][25]

Many journalists stated that the first "killer app" VR game was Half-Life: Alyx, developed by Valve and released in March 2020.[26][27][28][29] Alyx includes a number of novel control schemes to avoid the motion sickness problems of previous VR games, such as the 2019 indie title Boneworks.[30][31] Within a week of Alyx's announcement, Valve sold out of their stock of Index units and began taking pre-orders with expectations to fulfill before the game's release.[32][33] Other VR hardware, including the Oculus, saw increased sales leading to the release of Alyx.[34]

Hardware

An Oculus Quest 2 VR headset
An Oculus Quest 2 VR headset

Nearly all VR games, as generally defined, require the use of a VR headset that provides stereoscopic displays that simulate three dimensional reality and create immersion for the player. Most headsets include some means of positional tracking as to provide head-tracking (that is, as to tell which direction a player's head is looking), either through sensors built into the unit or from external sensors or cameras that are placed at the corners of the play area. Some headsets further provide eye tracking.[7] To provide immersive audio, either surround sound speaker systems are used, or headsets may be outfitted with speakers or headphones for the player that provide 3D audio effects.[7]

Some type of player input is also required. This is most commonly provided through the use of one or more game controllers. A controller can be as simple as a keyboard-and-mouse (KBM) or a standard game controller, or may be specialized hardware that includes positional tracking. Most often for specialized VR hardware, the player will have two controllers, one for each hand. These controllers may also provide haptic feedback to the user.[7]

Controls

Nearly all VR games are played from a first-person perspective as to take advantage of the level of immersion created by the headset. Player input, which includes the motion of their head, hands, and body and any controller buttons or triggers they depress, are translated into actions within the game. Most commonly, the player's view of the game's environment will track with how they move their hand, and they will be shown virtual hands to guide them how to interact with the environment, with VR translating the player's motions one-to-one with the virtual appendages.[7] While games can be controlled through traditional controls such as KBM or standard console controller, these interfaces break the level of immersion, and instead, more specialized controllers, typically designed to fit naturally into a player's hand.[7]

Gameplay of Job Simulator, a popular VR game, with the player's virtual hands manipulating the environment
Gameplay of Job Simulator, a popular VR game, with the player's virtual hands manipulating the environment

VR offers several novel control schemes - how the player manipulates their in-game character through the game world and the direction they are looking - compared to the traditional free look or mouselook offered by the traditional KBM or standard controller. Movement and aim may be coupled between the headset and the controllers or may be decoupled, typically with the headset controlling the direction of movement and controller the aim, which generally leads to more immersive experiences.[35]

Some VR systems such as the HTC Vive and Oculus offer room scale tracking, which not only incorporate the motions of the player but where they are physically located within a given area and the physical positioning of their body. This allows the player to move around the area as part of the VR experience. Games usually make this an optional experience since not all VR systems support it, and not all players have space to be able to move about. Example of games that support room scale tracking include Job Simulator and Rec Room.[36][37] When room scale is not available, alternate movement schemes have been developed when character movement is required. A player may be moved automatically by the game as necessary, a player may need to look at a target location and indicate through a control scheme their desire to move their character to that spot, or the player may use more traditional controls such as an analog stick or keyboard presses to move their character.

Design considerations

VR games are designed to enhance immersion—the perception that one is actually in the virtual world—and presence—the psychological effect that they are actually interacting with the virtual world outside of their physical bodies—concepts which cannot readily be done with traditional "flat screen" games played on a computer monitor or television.[38]

A limiting factor for VR games until the 2010s was the overall system latency between a player's actions and the feedback they saw on the headset. For VR to be felt as an immersive experience, the latency needs to be as small as possible so that the player sees feedback in as close to real-time to their actions. Technology bottlenecks had been from two major components of VR systems. One area was the rendering speed of computer hardware to update the 3D displays at a fast-enough frame rate. Frame rates of 20 Hz or less appear to most users as a series of separate images rather than continuous video stream, and breaks immersion.[39] In the late 1990s, this computational power could only reasonably been delivered by high-performance workstations such as those from Sun Microsystems and Silicon Graphics.[39] Since then, improvements in graphics processor technology and game engines with optimized rendering systems give consumer-grade hardware the capacity to perform high-speed real-time 3D rendering at 60 Hz or greater at resolutions appropriate for VR applications.[7]

The second bottleneck is the processing time to convert tracking sensor information into feedback that is incorporated into the game. Earlier VR systems took some time for complete acquisition of all tracking sensor information into usable feedback to the user, but this was at a longer timescale compared to the traditional inputs and the display feedback cycle.[39] Improvements have been made since in sensor technology and the software libraries to register movements, and VR games can also include other methods such as limited prediction of a player's movements, to bring sense feedback to the same timescale as rendering. Both issues combine to the overall factor of synchronization between the feedback loops. If the game takes too long to respond to a player's action, even if more than about 25 milliseconds, it further breaks the sense of immersion.[39] While many of the latency problems are resolved with the VR hardware of the 2010s, VR games still must be programmed with these concerns in mind.[39]

Other VR games

In its current meaning, "virtual reality" generally has been taken to creating immersion and presence with the player by creating a new visual stimulus (through a VR headset for example) that obscures to real world view.[40] This definition distinguishes VR from augmented reality where additional visual information is added atop the real world view.[7] A broader definition of virtual reality can be taken to be any application that replaces one or more of the human senses with a virtual one.[7] Thus, games featuring any alternative control scheme compared to a typical game controller or keyboard-and-mouse system could be considered as a virtual reality game, where the sense of touch of these traditional controls is replaced with a novel scheme. Such games would include those with alternate peripherals such as Dance Dance Revolution and Guitar Hero, or games featuring motion controls such as many Wii-based games.[8] However, with the expansion of VR hardware in the 2010s, the use of "virtual reality" to include these types of games has been deprecated.[7]

See also

References

  1. ^ a b Faisal, Aldo (2017). "Computer science: Visionary of virtual reality". Nature. 551 (7680): 298–299. Bibcode:2017Natur.551..298F. doi:10.1038/551298a.
  2. ^ a b Sturman, David J; Zeltzer, David (1994). "A survey of glove-based input". IEEE Computer Graphics and Applications. 14 (1): 30–39. doi:10.1109/38.250916. S2CID 7119184.
  3. ^ "Inside story on: The power glove (Cover)" (PDF). Design News. 45 (23): 63. 4 December 1989. Retrieved 16 December 2015.
  4. ^ a b Edwards, Benj (August 21, 2015). "Unraveling The Enigma Of Nintendo's Virtual Boy, 20 Years Later". Fast Company. Archived from the original on July 7, 2018. Retrieved December 21, 2015.
  5. ^ Wiltz, Chris (March 1, 2019). "The Story of Sega VR: Sega's Failed Virtual Reality Headset". Design News. Retrieved March 24, 2021.
  6. ^ Peternier, Achille; Cardin, Sylvain; Vexo, Frédéric; Thalmann, Daniel (2007). "Practical design and implementation of a CAVE environment". GRAPP 2007 - Proceedings of the Second International Conference on Computer Graphics Theory and Applications. Second International Conference on Computer Graphics Theory and Applications. INSTICC - Institute for Systems and Technologies of Information, Control and Communication. pp. 129–136. ISBN 978-972-8865-71-9.
  7. ^ a b c d e f g h i j k Coburn, Joshua; Freeman, Ian; Salmon, John (September 2017). "A Review of the Capabilities of Current Low-Cost Virtual Reality Technology and Its Potential to Enhance the Design Process". Journal of Computing and Information Science in Engineering. ASME International. 17 (3). doi:10.1115/1.4036921.
  8. ^ a b LaViola Jr, Joseph J. (2008). "Bringing VR and spatial 3D interaction to the masses through video games". IEEE Computer Graphics and Applications. 28 (5): 10–15. doi:10.1109/MCG.2008.92. PMID 18753029. S2CID 14083591.
  9. ^ Thomas, Bruce H. (2012). "A survey of visual, mixed, and augmented reality gaming". Computers in Entertainment. 10 (1): 1–33. doi:10.1145/2381876.2381879. S2CID 17464580.
  10. ^ Dan Kedmey (June 11, 2015). "Oculus Rift Headset Available Early 2016, Xbox Controller Included". Time.com. Retrieved July 17, 2015.
  11. ^ Facebook purchases VR headset maker Oculus for $2 billion Ars Technica, March 25, 2014
  12. ^ Nunneley, Stephany (December 12, 2018). "ZeniMax accepts settlement offer from Facebook in Oculus lawsuit". VG247. Retrieved December 12, 2018.
  13. ^ Webster, Andrew (June 11, 2015). "These are the first Oculus Rift games". The Verge. Retrieved July 17, 2015.
  14. ^ O'Brien, Lucy (June 11, 2015). "E3 2015: Square Enix, Harmonix Among Oculus' Publisher Partnerships - IGN". Uk.ign.com. Retrieved September 7, 2015.
  15. ^ Adi Robertson (8 December 2015). "HTC Vive VR headset delayed until April". The Verge. Archived from the original on 23 September 2017. Retrieved 23 September 2017.
  16. ^ "PlayStation VR Launches October 2016". Sony. Archived from the original on 22 July 2016. Retrieved 15 March 2016.
  17. ^ Machkovech, Sam (2019-04-30). "Valve Index reveal: The best of VR's first generation—but is it worth $999?". ArsTechnica. Conde Nast. Retrieved 9 May 2019.
  18. ^ Amadeo, Ron (March 3, 2021). "Google's VR dreams are dead: Google Cardboard is no longer for sale". Ars Technica. Retrieved March 24, 2021.
  19. ^ Heaney, David (April 5, 2020). "Facebook kills software updates for Samsung Gear VR". Venture Beat. Retrieved March 24, 2021.
  20. ^ Marshall, Cass (March 6, 2019). "Nintendo brings VR to Switch with new Labo kit". Polygon. Retrieved March 6, 2019.
  21. ^ https://www.youtube.com/watch?v=V5VI-fpVqhA. ((cite web)): Missing or empty |title= (help)
  22. ^ Sherr, Ian (September 24, 2018). "VR's missing link? A killer app that convinces us to buy in". CNet. Retrieved March 24, 2021.
  23. ^ Von Camp, Jeffery (January 11, 2018). "At CES, Everyone Is Still Hunting for VR's Killer App". Wired. Retrieved March 24, 2021.
  24. ^ Anderson, Mae (January 9, 2019). "Remember virtual reality? Its buzz has faded at CES 2019". Phys.org. Retrieved March 24, 2021.
  25. ^ Jenkins, Arik (June 20, 2019). "The fall and rise of VR: The struggle to make virtual reality get real". Fortune. Retrieved March 24, 2021.
  26. ^ Carbotte, Kevin (23 March 2020). "Half-Life: Alyx Gameplay Review: (Almost) Every VR Headset Tested". Tom's Hardware. Retrieved March 24, 2020.
  27. ^ Robinson, Andrew (March 23, 2020). "Review: Half-Life Alyx is VR's stunning killer app". VGC. Archived from the original on March 24, 2020. Retrieved March 24, 2020.
  28. ^ Oloman, Jordan (23 March 2020). "Half-Life: Alyx is a watershed moment for virtual reality | TechRadar". www.techradar.com. Archived from the original on March 24, 2020. Retrieved March 24, 2020.
  29. ^ "CES 2020: Teslasuit Will Unveil New Haptic VR Gloves". Tech Times. December 27, 2019. Archived from the original on March 25, 2020. Retrieved March 24, 2020.
  30. ^ McKeand, Kirk (April 7, 2020). ""The Boneworks thing is weird" – Valve says it took more inspiration from Budget Cuts for Half-Life: Alyx". VG247. Archived from the original on April 7, 2020. Retrieved April 7, 2020.
  31. ^ Brown, Fraser (2020-04-07). "Half-Life: Alyx's movement system was changed because tall testers kept getting stuck". PC Gamer. Retrieved 2021-03-13.
  32. ^ James, Sarah (November 29, 2019). "Valve's Index VR headset sold out in stores following the Half-Life: Alyx reveal". PC Gamer. Archived from the original on November 30, 2019. Retrieved December 1, 2019.
  33. ^ Nunneley, Stephany (February 21, 2020). "Coronavirus has affected the production schedule for Valve Index, fewer units to be made available". VG247. Archived from the original on February 21, 2020. Retrieved February 21, 2020.
  34. ^ Robertson, Adi (April 30, 2020). "Oculus sales spiked in the lead-up to Half-Life: Alyx". The Verge. Retrieved March 24, 2021.
  35. ^ Martel, Erin; Su, Feng; Gerroir, Jesse; Hassan, Ahmed; Girouard, Audrey; Muldner, Kasia (2015). Diving Head-First into Virtual Reality: Evaluating HMD Control Schemes for VR Games. Proceedings of the 10th International Conference on the Foundations of Digital Games (FDG 2015). Society for the Advancement of the Science of Digital Games.
  36. ^ Takahashi, Dean (February 26, 2017). "How Owlchemy Labs designed VR's funny hit Job Simulator". Venture Beat. Retrieved March 24, 2021.
  37. ^ Ackerman, Dan (August 12, 2016). "VR finally has its killer app, and it's called Rec Room". CNET. Retrieved December 28, 2017.
  38. ^ Yildirim, Caglar; Carroll, Michael; Hufnal, Daniel; Johnson, Theodore; Pericles, Sylvia (2018). Video game user experience: to VR, or not to VR?. 2018 IEEE Games, Entertainment, Media Conference. IEEE. pp. 1–9.
  39. ^ a b c d e Hubbold, Roger; Murta, Alan; West, Adrian; Howard, Toby (1995). "Design issues for virtual reality systems.". In Göbel, Martin (ed.). Virtual Environments' 95. Springer. pp. 224–236. ISBN 978-3-7091-9433-1.
  40. ^ Steuer, Jonathan (1992). "Defining virtual reality: Dimensions determining telepresence". Journal of Communication. 42 (4): 73–93. doi:10.1111/j.1460-2466.1992.tb00812.x.