Where is the future of augmented reality heading?

• Geospatial mapping and Shared AR are transforming gaming experiences, while also showing promise for everyday applications.
• Generative AI is revolutionizing the way content is created in augmented reality.
• There are use cases for geospatial mapping and AR from gaming to enterprise solutions.

In an accelerating digital landscape, augmented reality (AR) is transforming human-computer interaction, with examples spanning from gaming experiences like Pokémon GO to innovative enterprise solutions.

Recently, Tech Wire Asia spoke with Brian “Bam” McClendon, Senior Vice President of Engineering at Niantic. The conversation covered a range of topics, including geospatial mapping and the ethical complexities of this burgeoning field.

But how exactly does Niantic plan to achieve this integration of AR into everyday life? McClendon provided some insights through the lens of geospatial mapping and shared experiences.

Geospatial mapping meets shared experiences: Augmented reality examples from Niantic

When Niantic launched Pokémon GO in 2016, it not only captivated gamers but also showcased the untapped potential of geospatial mapping. The game’s “location AR” feature allows players to navigate a digitally-enhanced real world, underlining the critical importance of precise visual positioning and 3D real-world models.

But Niantic’s aspirations extend beyond solo experiences. McClendon stressed the value of Shared AR, exemplified by communal in-game events such as the sudden appearance of a Snorlax in Pokémon GO. In these scenarios, collective augmented reality enriches the gameplay by creating a synchronized, interactive experience for the community.

This notion of communal interaction is further accentuated by the implementation of 8th Wall, which offers players not only a shared digital canvas but also the ability to see each other’s interactions within it.

While the fusion of geospatial technology and communal experiences is promising, it’s not without its hurdles. McClendon offered his views on the challenges that lie ahead.

The complex landscape of app discoverability and challenges in AR game development

The app-based gaming ecosystem presents its own set of obstacles, primarily around discoverability. McClendon contends that one of the most formidable challenges in gaming today is gaining visibility within an app store that is flooded with millions of competing apps.

“Unless you invest significant amounts of money in marketing or get lucky, your ability to reach users is severely limited,” he explained. He criticizes the gatekeeping roles played by Apple and Google and contrasts this with web-based discovery, which offers multiple avenues like search engines, QR codes, and word of mouth.

In this context, he sees web-based platforms like 8th Wall as democratizing forces, offering a wider variety of 3D, AR, and VR gaming experiences that are more readily discoverable and shareable.

In the realm of AR game development, McClendon discussed a variety of challenges, ranging from content creation to scalability. “High-quality artists are essential,” he noted, and talked about the strategic placement of characters and objects within the game.

He mentioned the promising capabilities of Generative AI for content creation. “One of the fascinating aspects of generative AI is its capability to prototype, and in some instances, fully produce parts of the content,” he said. In particular, he mentioned characters in Inworld, which can be animated “in the action sense, not just the 3D sense—using large language models.”

Moreover, new technologies like Stable Diffusion, Midjourney, and DALL·E are enhancing creative possibilities.

Another challenge McClendon highlighted was scalability. “This must be built on a stable platform that can scale from a thousand users to a hundred million,” he said, acknowledging that Niantic has the expertise to scale gaming platforms to meet the demands of hundreds of millions of users.

He also underscored the importance of creating a “persistent experience” that can be shared across a large user base.

But these challenges aren’t solely confined to the app ecosystem. McClendon also highlighted some fascinating technological solutions, particularly the role of Generative AI in AR.

Generative AI: A game-changer in augmented reality content creation

Generative AI has emerged as a crucial tool in deploying geographically and contextually relevant content at a global scale. For McClendon, one of Niantic’s key learnings is that building a real-world augmented reality game that can be played globally presents unique challenges.

“With Pokémon GO, for example, we deployed a carefully selected set of Pokémon with affinities to different geographical locations. While that algorithm has been effective and we’ve invested significant time into its development, it doesn’t offer the level of precision needed for more geometrically and geographically specific deployments,” he explained.

He gave an analogy: if you want to create a virtual golf course that perfectly fits into a town square, for instance, it has to be precisely sized and oriented. Manually authoring such content on a global scale is not feasible. You might be able to tailor experiences for Times Square or Shinjuku, but you can’t manually cover every possible location.

Generative AI can help deploy geographically relevant content that is customized to specific locations. For instance, if the location is Shinjuku, with its bustling intersections and numerous screens and crosswalks, you can input these details into the generative AI algorithm to create content that is contextually appropriate.

Because Generative AI operates at the scale of computation, McClendon emphasized that it can accomplish this task for every location, globally.

If you were to ask game content creators to achieve this, they might handle 10, 100, or even 1,000 locations – with a lot of effort. “However, they simply couldn’t cover 50 million locations around the world. That’s why one of the most promising applications for generative AI is in creating geographically customized content,” he added.

While Generative AI presents a solution to some challenges, the question of how to measure the success and ethical considerations of AR remains. McClendon weighed in.

Metrics, ethics, and augmented reality: What lies ahead?

In the evolving AR industry, McClendon explained the metrics and challenges affecting the technology’s adoption. Notably, he pointed to the mobile phone sector as a significant avenue for immediate business opportunities in AR. “While some phones and locations perform better than others, the sheer volume of the installed base presents an enormous business opportunity,” McClendon said.

Although promising prototypes for devices like Quest 3 and Apple’s Vision Pro are on the horizon, McClendon said the mobile sector will continue to be pivotal for the foreseeable future. With new daily-wear outdoor AR glasses coming to market, intriguing applications can be expected.

While data privacy is certainly at the top of McClendon’s ethical concerns—emphasized by Niantic’s careful handling of location data and upcoming precautions in AR headsets—this is just one facet of a broader ethical landscape in augmented reality.

McClendon acknowledged that the company’s focus has primarily been on privacy: “Location privacy, for example, has always been at the core of our company, especially with games like Pokémon GO. The most significant privacy risk comes from companies that share user data beyond its intended purpose,” he emphasized. Niantic takes extra steps to align its approach with preserving user privacy, ensuring that photos taken in private settings are not accessible to third-party applications.

However, ethical considerations in AR extend beyond just privacy issues. Accessibility is another point of concern; not everyone can afford the smartphones or headsets required to engage with AR technologies, raising questions about digital inequality. As AR becomes increasingly integrated into public spaces through geospatial mapping, issues related to safety and potential trespassing also come into focus.

Moreover, as the field increasingly relies on advanced algorithms like Generative AI for content creation, ethical scrutiny over algorithmic biases and fairness is warranted. “As we venture into using AI technologies to augment reality, it’s vital to consider the ethical implications, from the datasets we use to the experiences we create,” McClendon said.

By tackling these varied ethical challenges head-on, McClendon explained that the industry can work toward a more inclusive and responsible AR ecosystem.

Beyond gaming: Examples in enterprise and everyday life

While AR’s applications in gaming are groundbreaking, McClendon pointed out that its impact reaches much further. Let’s delve into how Niantic is applying AR technologies in non-gaming sectors.

Niantic is about much more than gaming. The company has developed a range of technologies including semantics, visual positioning, and depth sensing, and offers these capabilities through two key platforms: Unity and 8th Wall. Unity developers can utilize Niantic’s AR Development Kit (ARDK) along with Unity’s own AR Foundation.

In contrast, 8th Wall provides a web-based platform that doesn’t rely on Unity, AR Foundation, or AR kits from Apple and Google—ARKit and ARCore, respectively. “As a result, we’ve had to build many of those features from scratch to make AR possible on 8th Wall,” said McClendon. 8th Wall aims to establish a standard for web-based AR capabilities.

So, outside of gaming, the company’s AR technologies are also being deployed in non-gaming contexts. “We’re committed to a vision that goes beyond gaming,” said McClendon. The 8th Wall portfolio demonstrates this with hundreds of example applications that range from marketing experiences and product demonstrations to visual product navigation systems. For example, users can place a virtual piece of furniture in their room to see how it fits, or walk around a 3D model of a car to explore its features.

“Though the gaming industry often drives technological advancements, once those technologies are refined, they become valuable tools for enterprise applications as well. While gaming is an important aspect of what we do, it’s by no means the limit. We see a future that is much broader, encompassing various sectors and use-cases,” McClendon added.

As we delved deeper into Niantic’s future plans, McClendon spoke about a unique technology that could very well revolutionize our understanding of real-world navigation.

One of Niantic’s major focuses is the Visual Positioning System (VPS), a technology that enables users to determine their exact position in the physical world with far greater accuracy than GPS. Unlike other visual positioning systems, which offer accuracy within one or two meters, Niantic’s VPS allows for even finer precision.

“Currently, our VPS covers hundreds of thousands of locations,” McClendon said. “Developers can preview these locations before they physically visit. This means they can virtually explore a location or design an AR experience tailored for a specific site. Moreover, if a location is already VPS-enabled, content can be deployed there seamlessly. Our aim is to emphasize procedural deployment, allowing developers to create content once and easily deploy it across multiple locations worldwide.”

McClendon said he believed that this level of precision and versatility could redefine real-world experiences, particularly for pedestrians.

As AR technologies continue to mature, the transformative impact that McClendon emphasized—spanning from gaming to enterprise solutions—is becoming increasingly palpable.

---

---

---

---