What is spatial computing?

Around the world, billions of people use spatial computing applications every day. Yet many of us still don’t understand what spatial computing is or how it works. Similar to other terms, such as natural language processing or deep neural networks, spatial computing is a term that is often overlooked as some complex aspect of technology. However, it is true that the term actually defines the way we interact with our digital landscape.

Whenever you use a ride-sharing app, GPS, social media location marking, or even an AR app like Pokémon Go, you’re part of the spatial computing revolution.

Scientists, health experts and researchers rely on spatial computing technology to track diseases and map key information about the ocean floor. Environmentalists use spatial computing to determine the behavior of extinct species, and self-driving cars use spatial computing to safely bring passengers to their destinations.

Spatial computing is not only on track to profoundly change our lives, but has significantly impacted the way we live.

What is spatial computing, or spatial computing?

Spatial computing refers to the process of using digital technology to make computers communicate seamlessly in a three-dimensional world, combining refined reality (AR),virtual reality (VR) and mixed reality (MR). Spatial computing uses physical space to send inputs and receive outputs from the computer.

The term “spatial computing” was defined by MIT’s Simon Greenwold of the Media Lab in his very futuristic thesis in 2003. However, only recently have we been able to realize his thesis and vision, due to advances in technologies such as artificial intelligence (AI), camera sensors and computer vision that monitor environments, people and objects, IoT (Internet of Things) that monitors and controls products and assets, and refined reality (AR) that provides a human user interface.

At the highest level, spatial computing is the virtualization of activities and interactions between machines, people, objects and environments in which they take place to enable and optimize actions and interactions. In other words, it adds knowledge of relative location, i.e. location relative to other locations, in order to expand the concept of ‘traditional computing’. For example, an autonomous vehicle uses GPS, LiDAR, volumetric camera sensors and other technologies to determine its precise location and measure proximity to objects in a driving environment.

A practical example

Imagine Ivana, an 80-year-old who lives independently and uses a wheelchair. All items in her home are digitally cataloged; all sensors and devices that control objects have access to the Internet, and the digital map of her home is connected to the map of objects. As Ivana moves from her bedroom to the kitchen, the lights turn on and the ambient temperature adjusts. A wheelchair will slow down if her pet crosses her path. When she reaches the kitchen, the table moves to make it easier for her to access the refrigerator and stove, then moves back when she’s ready to eat. Later, if she starts falling as she gets into bed, her furniture moves to protect her, and the warning goes to her son and the local monitoring station.

In the field of medicine, consider this scenario: a paramedic team is sent to a city apartment to take care of a patient who may need urgent surgery. While the system sends the patient’s medical records and real-time updates to paramedics’ devices and to the emergency department, it also determines the fastest driving route to the person. Red lights stop traffic at intersections. As an ambulance approaches the building, the front door opens, revealing the elevator waiting for them. Objects move out of the way as paramedics rush inside with their stretchers. As the system takes them to the emergency room the fastest route, the surgical team uses spatial computing and refined reality to plot the choreography of the entire operating room or plan a surgical path through the patient’s body.

Spatial computing is the next step in the constant convergence of the physical and digital worlds. It does everything virtual and refined reality applications do: it digitizes objects that connect via the cloud, allows sensors and motors to react to each other, and digitally represents the real world. It then combines these capabilities with high fidelity spatial mapping to allow a computer “coordinator” to monitor and control the movement and interactions of objects as a person moves through the digital or physical world. Spatial computing will soon bring human-machine and machine-machine interactions to new levels of efficiency in many segments of life, among them industry, healthcare, transportation, and home. Large companies, including Microsoft and Amazon, are investing heavily in that technology.

Examples of spatial computing devices

Although the concept has become increasingly popular lately, several spatial computing devices have been in use for some time.

1. VR headsets

VR headsets are devices that allow users to experience part of the virtual world. In the past, they were used by players to interact with objects and video game characters. Recently, their use has evolved to include other applications, like training and simulation.

For several years, many companies have been improving the way they develop wearable VR headsets that allow users to interact with the digital world. Some of the popular VR headsets on the market are Valve Index, HTC Vive and HP Reverb G2.

2. AR glasses

AR glasses allow users to go deeper into the digital realm. These devices project data and images and are especially useful in industrial applications. Some of the AR glasses worthy of attention on the market are NuEyes, Microsoft HoloLens and NReal Light.

3. Hybrid equipment

Hybrid equipment uses VR, AR and MR technology. This app allows the user an immersive experience as they can fully integrate their senses. While such products are still in the development phase, they could soon become a reality with big tech companies, like Samsung, Google, Apple and Microsoft, funding startups to create cutting-edge hybrid equipment. The best example is the FellReal multi sensory mask.

Advantages of spatial computing outside of games

The biggest advantage of spatial computing is the ability to perceive digital content as part of reality. It has specific advantages that decades ago we could only imagine. Now it has become a “spatial” reality.

More interactive staff training

Realistic three-dimensional (3D) visualization provided by spatial computing makes employee training more interactive and interesting. It allows workers to test a new feature, product, or service without waiting for development to end. This gives companies an advantage, as their employees would be well acquainted with the product even before it hits the market.

Lower product development costs

Since employees can communicate with a new product or service through spatial computing, they can get acquainted with them in advance. Companies can therefore save on costs as they do not have to rush to develop the final product just for the sake of training staff.

More accessible real estate tours

AR built into spatial computing makes virtual 3D tours feasible for any property. Whether it’s in the sale, construction or design phase itself, this type of real estate tour provides potential clients with a detailed overview regardless of their location. Of course, this option is also relevant for interior designers, engineers and construction entrepreneurs.

Spatial computing and interaction technology

As visual content becomes more authentic and three-dimensional, the way we communicate with it must follow the same example. Handheld controllers will always have a place in home gaming and for expert users. But the broad adoption of spatial computing (looking beyond games into industrial design, training, healthcare, theme parks, and education) will depend on natural interaction.

Hand tracking

Tracking, and especially hand tracking, means you can interact directly with virtual content. You can grab, squeeze, push, slide, and slide virtual objects directly, without having to learn to press buttons or keyboard shortcuts.

When you take an object in the physical world, it is instantaneous and effortless. The same must be true for spatial computing. Hand tracking software must be able to display hands realistically, in real time with high fidelity and low latency.

Perceptibility

Imagine how hard it is to button a button while wearing a pair of gloves. This gives you a hint of how important your sense of touch is to natural interaction.

This makes haptics another key interaction technology for spatial computing. Haptic feedback in AR/VR can be created using wearable materials (haptic gloves, vests and suits). A prime example is the newly released Teslasuit. While we’re far from being able to digitize touch as a whole, even small touches of feedback dramatically improve the user experience.

Careful weaving of haptics into AR/VR interfaces is especially important for high-end or business products where high-quality precise interactions are a key feature.

Voice controls

The use of language is almost as instinctive as using our body. The number of voice digital assistants in the world will exceed the entire human population in 2021. The ability to engage in AR or VR by using body and voice creates exponential possibilities. The classic 1979 MIT experiment, Put That There, it illustrates it nicely.

Eye tracking

View tracking technology, while mainly concerned with delivering effective visual representations, can also be used to increase interactivity effortlessly. Hedset manufacturers such as Varjo use their ability to track where the user is looking to better understand intent. This allows them to extract relevant, contextual information such as interaction menus in a fluid way effortlessly. It minimizes clutter and reduces cognitive load.

Why haven’t we fully adopted spatial computing yet?

If spatial computing is so amazing, and technology has begun to be produced by a larger number of companies, why is it not yet commonplace in our society? While we have some aspects of AR and VR that we can enjoy in the current environment, from smartphone apps to gaming hedsets, we haven’t adopted nearly as much of this technology as we could.

The biggest problem, as with most new technologies in the current environment, is that they need to be adopted. There are many challenges involved in getting anyone to adopt and apply the new technology. This is especially true in large social spaces and companies that already have a certain way of working. Just as many companies in the current world struggle to move away from hardware commitments to software and the cloud, people also have a hard time accepting spatial computing.

As with any other digital transformation attempt, companies will need to take the time to define the benefits of introducing more spatial computing solutions into the workplace. Once companies of all shapes and sizes begin to feel more comfortable stepping into a digital landscape with spatial computing, acceptance throughout the world will grow. The more this happens, the more developers in the spatial computing community will be able to invest in new technology that is changing the way we work.

Furthermore, it is worth noting that much of spatial computing technology today still requires different pieces of different technology and information that need to be merged into a more unique environment. A lot of expertise is needed to make spatial technology as impressive as it needs to be. We’re still working to make the hardware and software perfect. As a result, there is still a lot of work to be done.

Today we are just on the verge of diving into the landscape of spatial computing. We started immersing our fingers in the water, exploring the possibilities that can come from smartphone apps and gaming headphones. However, in the years to come, as connections become more efficient with 5G and the technology evolves further, the capabilities of this technology are bound to increase.

We have already shown that we can accept the value of spatial computing with things like smart assistants and AR applications. It will be interesting to see where this new age of augmented reality can take us further.