Apple Inc. is seeking a breakthrough product to succeed the iPhone, and aims to have technology ready for an augmented reality headset in 2019 that could ship as early as 2020.
Unlike the current generation of virtual reality headsets that use a smartphone as the engine and screen, Apple’s device will have its own display and run on a new chip and operating system, according to people familiar with the situation. The development timeline is very aggressive and could still change, said the people, who requested anonymity to speak freely about a private matter.
While virtual reality immerses the user in a digital world, augmented reality overlays images and data on the real one. CEO Tim Cook considers AR less isolating than VR and as potentially revolutionary as the smartphone. He has talked up the technology on Good Morning America and gives it as almost much attention during earnings calls as sales growth.
“We’re already seeing things that will transform the way you work, play, connect and learn,” he said in the most recent call. “Put simply, we believe AR is going to change the way we use technology forever.”
Apple declined to comment.
Apple began putting together a team to work on AR-related projects a couple years ago, Bloomberg reported in March. Led by Mike Rockwell, who previously ran engineering at Dolby Labs, the group has now grown to several hundred engineers from across Apple, the people said. Scattered across office parks in both Cupertino and Sunnyvale, California, the team is working on several hardware and software projects under the umbrella code name of “T288.”
The team’s first product was ARKit, tools that outside software developers use to create AR applications for the latest iPhones and iPads, leveraging their screens, cameras and processors to create virtual 3D interfaces for online shopping, education and gaming. This was an interim step, giving Apple an opportunity to test the technology on an existing product.
The next step — creating a headset with a built-in display capable of streaming 3D video without draining the battery — is much more complicated. Cook acknowledged as much in a recent interview with The Independent, when he said: “Anything you would see on the market any time soon would not be something any of us would be satisfied with.” Referring to challenges creating displays, Chief Design Officer Jony Ive told a tech panel last month that “there are certain ideas that we have and we are waiting for the technology to catch up with the idea.”
As with previous products, Apple isn’t waiting around for someone else to create a chip capable of powering its AR headset. It’s designing one in-house that’s similar in concept to the “system-on-a-package” component in the Apple Watch. Such chips can squeeze more components — graphics processors, AI chip, CPU — into a smaller area than standard processors; they also consume less power.
The new operating system, internally dubbed “rOS” for “reality operating system,” is based on iOS, the iPhone’s operating system. Just as tvOS powers the Apple TV, macOS runs on Macs and watchOS runs on Apple Watches, “rOS” will power Apple’s AR headset. Geoff Stahl, formerly a software manager for games and graphics at Apple, is one of the directors of the “rOS” software group.
Apple hasn’t finalized how users will control the headset and launch apps, but is investigating touch panels, voice-activation via Siri and head gestures. Engineers are prototyping a range of applications, from mapping and texting to more advanced features including virtual meeting rooms and 360-degree video playback. The company has discussed pairing the headset with its own version of the App Store, where users would be able to download content, just as they do with the iPhone, Watch, Apple TV and Mac.
Because Apple doesn’t have a fully operational headset of its own, engineers have begun using HTC Vive headsets for testing purposes. They’re also working on a device similar to an Oculus Gear VR headset that uses an iPhone’s screen, cameras and chipsets. Apple doesn’t plan to sell the gadget but instead aims to use it internally to test AR apps next year.
With the headset at least two years away, Apple wants to make it easier for developers to bring new AR features to the iPhone. To that end, the company plans to release a new version of ARKit software tools as soon as 2018. They could help developers create software that remembers accurately where a digital object was placed in a virtual space, using a technology called persistent tracking; you look away, and the object is still there. The tools will also make it easier to create AR games for multiple players.
By Mark Gurman
