Extended Reality

What is XR?

XR or "Extended Reality" is an umbrella term used to describe technologies that blend digital experiences with physical reality. While many innovative XR technologies are becoming increasingly mainstream, some of the ideas underlying VR and AR technologies can be traced back to the 1800s and stereoscopic displays, which play with vision. These early devices made it possible to capture and represent the world in 3D images using mirrors and refracting prisms. The Crash Course: Future of VR episode has a more detailed look at the history of VR (video includes transcript).

A timeline for immersive technologies from 1962 to today.

What's the difference between AR, VR and MR?

Technologies that fall under the term XR provide us with a more immersive way to interact with digital objects. What do AR, VR, and MR mean? 

Virtual Reality (VR) usually relies on a head mounted display with screens for each eye. This allows content creators to generate “stereoscopic” video which translates into a three-dimensional experience. VR devices can immerse a user in real-world 360 video, computer-generated content, or a combination of both; these are usually controlled by handheld remotes. Examples of VR include the video game Second Life, the Mozilla Hubs platform of virtual rooms, or, in fiction, the Star Trek Holodeck. 

With Augmented Reality (AR) experiences, digital content is layered on top of the physical world using headsets, smartphones or wearables. With AR, the wearer’s environment can be augmented with text, images, video, sound, and other forms of haptic feedback – just like in Pokémon Go. Other examples include Tiktok’s animated overlays, or Ikea’s app which helps you visualize furniture in a room before buying it. 

The defining feature of Mixed Reality (MR), and what differentiates it from AR, is that objects in the physical world and digital content coexist and interact with each other, and with you! For example, Google Translate can overlay translated text over real life script. If you change what is written, the translation will update. 

Generally, XR can be thought of as any system which blurs the lines between what is real and what is computationally simulated by making the latter interactive and immersive.

Graphics license CC-BY for the Noun Project, by Arafat Uddin, Path Lord, and Smashicons.

How will XR affect our every day lives?

For some people, XR is already affecting the way they work or live. As mentioned above, translation apps use AR to translate text. Digital maps often let you overlay information over a satellite representation of the terrain surrounding you. Book scanning apps detect the edges of a page and display those in real time on your mobile device to make sure you will get the output you want. 

In healthcare, VR helps people get in the shoes of people who perceive the world in different ways or to prepare patients for medical procedures. In retail, brands have, like Ikea, used AR or VR to let curious customers “try out” products before purchasing. In digital arts, some people have reoriented their career towards exploring the artistic potential of these tools, creating immersive experiences, compositions, games, etc. 

At research centers, XR technologies are also being used to visualize and explore large datasets. For example, NASA’s PointCloudsVR is used to map and explore star system information gathered by orbital satellites. Finally, various museums provide high resolution tours of their collections in immersive formats, making their collection remotely accessible.

NASA engineer Tom Grubb manipulates a 3D simulation that animates the speed and direction of four million stars in the local Milky Way neighborhood using the PointsCloudVR system. CC-By Wikimedia Commons.

Single and Multi-Player Environments

In the 1990’s, VR environments tended to be single-player worlds. Projects such as Second Life (released 2003) popularized the possibility of interacting with other users via avatars in a virtual world, which have since become the norm, and allow for further integration with AR / MR experiences. 

In 2021, Facebook changed its name to Meta and publicized its version of a Virtual Reality platform. Learn more from the Web 3.0 and the metaverse Quick Thing.

A Meeting in Second Life, one of the original multi-player commercial online meeting environments. CC-BY Wikiversity.