[ad_1]
Apple is about to release its most important new product in a decade. According to rumors, Reality Pro will be a unique virtual headset. No, I mean AR headset. Wait, do it with an MR headset dial switch between AR and VR at will…
Despite years of headset releases from big-name corporations such as Google, Microsoft, Sony, Facebook/Meta and others, most people still don’t understand AR, VR or MR. Before Apple brings Reality Pro to our faces, let’s be clear on what that means and discuss what we can expect from Apple’s new platform.
A virtual reality
Let’s start with the easy one. Virtual reality (VR) products place screens with lenses in front of your eyes to expand their field of view. There are usually two screens (or one split in half) each displaying the point of view of each of your eyes to create a true 3D effect.
The movement and position of the headset must be tracked using gyroscopes and sensors to correct your gaze with every little movement of what you see on the screen. In order for you to feel completely immersed and avoid motion sickness, this must be done very quickly, with a very low “motion delay to photon” (the time between moving your head and this change being shown on the displays).
The end effect is that you look like you’re in a completely computer-generated place. It may be cartoonish or realistic, but everything – the environment and everything in it – is made of computer graphics. This is virtual reality: no more real world, all virtual world.
Product examples: Meta Quest, PlayStation VR, HTC Vive, Valve Index
Meta
augmented reality
Augmented Reality (AR) does not replace the real world with a virtual one, but rather incorporates digital objects into the real world. This can be done on the screen, for example, in the augmented reality app on your iPhone, in a headset, or with glasses.
The main idea is that you see the real world, but computer generated. things added to it: icons, arrows, text, floating video screens, people, whatever.
An important note is that the graphics should look like be present in the real world. An AR object can stay in the same place in the real world as you move around it, and can even interact with real world geometry such as the ground, tables, walls, or even people. Simply displaying floating graphics on a transparent display in front of your eyes is not augmented reality.
AR requires at least some rudimentary 3D rendering of your immediate surroundings, which is usually done using multiple cameras and sensors such as LIDAR. Many products market themselves as AR, but are more accurately referred to as “windshield screens” because they cannot actually integrate computer graphics into the real world. These products don’t create a 3D map of your environment to integrate graphics into, they simply move objects in front of you. For example, Google Glass, Nreal Air and Vuzix Blade 2.
Pokemon Go often referred to as an AR app and this is often confused with the fact that you are playing it in the real world. It’s simple location based games, not AR. The game has one small part of the augmented reality game: when you throw your pokeball to catch a pokemon, you can turn on the “augmented reality mode”, which shows your target pokemon in the real world on the screen of our phone. This part, and this part only, qualifies as an AR. (And most players turn it off.)
An AR headset can look just like a VR headset, completely blocking your view of the outside world if it uses external cameras to stream real-time video to the screens in your surroundings. This is how Apple Reality Pro is expected to work, and it contrasts with something like Microsoft’s HoloLens, which has a transparent screen with a built-in display so you actually see the real world itself.
However, as long as you see the real world around you in real time with the help of computer graphics integrated into it, and not just hovering in front of you, this is considered augmented reality.
Product examples: Microsoft HoloLens, Magic Leap, and the upcoming next-generation Snap Spectacles are considered real glasses or augmented reality headsets.
Microsoft
mixed reality
Here is where it gets confusing. Apparently, augmented reality is not augmented. enough so the industry has come up with a term that can’t even be clearly defined to separate real AR from all these virtual displays and heads-up displays, as well as from AR apps for phones.
Look at mixed reality (MR) and you’ll find a dozen different definitions full of not-so-precise phrases such as “brings the real world and the virtual world together in a more immersive way” or “allows multiple users to interact.” together in virtual space. Some say that you need to manipulate virtual objects with your hands, while others say that mixed reality can use controllers.
Intel says MR “brings together the real world and digital elements” (which is what AR does). “It gives you the ability to stand with one foot (or hand) in the real world and the other in an imaginary place, breaking down core concepts between the real and the imaginary, offering an experience that could change the way you play and work today.” Poetic, but not particularly helpful.
Meanwhile, Microsoft says, “It frees us from a screen-tethered experience by offering an instinctive interaction with data in our living quarters and with our friends.” I don’t know what “instinctive interaction” with data is, but it sounds incredibly boring. “People may not even realize that the augmented reality filters they use on Instagram are mixed reality.”
Thus, even among the largest tech companies, there seems to be no consensus on how to define MR or what minimum standards allow a product or experience to be called mixed reality.
I think a good way to think about it is this: mixed reality is superset augmented reality. This is augmented reality with a higher minimum standard of interaction and immersion. It is viewed with one’s own eyes (either through transparent glasses or through a headset with pass-through video from external cameras), and not displayed on the screen of a phone or tablet. And it doesn’t just display information integrated into the real world, it allows you to interact with these virtual objects and the objects to interact with the environment.
What to expect from Apple Reality Pro
From what we know about Apple’s headset, it will provide an unrivaled VR, AR, and MR experience. It is said to be a headset not unlike a pair of ski goggles that will hide your view of the outside world.
Several cameras will display the world around you in real time, and a set of sensors will create a three-dimensional map of your environment so that virtual objects can integrate and interact with it. You’ll control apps and manipulate objects with your hands, and internal sensors will even detect your eye movements.
The dial on the outside, similar to the digital crown on the Apple Watch, will let you control which part of the outside world you want to see. Turn it to one side and the world will be blocked, leaving only virtual environments: VR. Turn it the other way and you’ll see the full real world (shown on headset displays from multiple external cameras) with integrated computer graphics (AR/MR).
We’ll know more about Reality Pro’s features when it’s officially announced, likely on June 5th at the WWDC programming conference. If the current rumors are true, it will set a new high bar for quality, detail, immersion, interaction… and price, perhaps up to $3,000.