Future Tech 2025: An Interview with Scott O’Brien, Augmented Reality OMG

In this Future Tech interview, we’re speaking with Scott O’Brien, Founder of Augmented Reality OMG. As an Augmented Reality (AR) pioneer since 2009 and delivering what is still to this day, the worlds most sophisticated cinematic AR eyewear experience, Scott O’Brien has been moving AR from “magic show” to functional use cases and mass adoption. These days he consults worldwide under Augmented Reality OMG and organises AR, VR, wearables and IoT events through Reality Remix primarily in Asia Pacific.

Shara Evans (SE): This is Shara Evans, CEO of Market Clarity. I’m in Sydney, speaking with Scott O’Brien, one of the early pioneers in the field of augmented reality. He’s the founder of a company called Augmented Reality OMG. He splits his time between here and the US, and I’m lucky to have managed to spend some time with him this afternoon, talking about the directions of AR.

Welcome, Scott.

Scott O’Brien (SO): Thank you for having me.

SE: Why don’t you share with us a little bit about your background and how you began your journey into the world of augmented reality?

SO: Well, through playing sport with a friend who was lecturing in augmented reality, it took me about 60 seconds to realise the magic of it. This was in 2009. I was helping an event called Online Retailer, and it was all about going beyond picture and price for retail experiences. Having seen augmented reality and how it could help in trying clothes, and jewellery, visualise products, more or less try products in your hand without the physical constraints, I immediately saw the opportunity and, within six months, started a new company. It was called Explore Engage. It helped people explore their world and more or less engage in their dreams. We operated for a good four years, doing about a hundred projects.

SE: A lot of people reading this may not know exactly what augmented reality is. Can you give us just a quick explanation of what you mean when you describe augmented reality?

SO: Well, in some ways I then need to explain: what is reality? Augmented reality is a digital overlay of the real world. Usually, it’s intelligent and contextual overlay. It could be based on the image that you’re looking at, on your geospatial position, or some other reference point to provide an improved experience of your surrounds or an object — for example, packaging — that you’re looking at.

SE: Give us an example. If I’m looking at a package on a shelf in the supermarket, what might augmented reality show me that’s different from what I see with my own eyeballs?

SO: Well, a lot of people these days, celiacs, gluten intolerants, etc, when they go shopping in the grocery aisles, they want to be able to readily identify what foods they can buy. By scanning the product or barcodes they’ll be able to get this information in real time and shop with ease and confidence. It’s a great risk management tool for the retailer but also for consumers.

Beyond dorky glasses

SE: A lot of people, when they think of augmented reality, they think of dorky-looking glasses, but in fact we can use our smartphones; we can use different kinds of eyewear, and there may even be in-store displays that can be used with an augmented reality twist. What’s your take on all of that? How do these devices work?

O: Well, no doubt, fashion is an aspirational quality for augmented reality, but initially we have a very strong parallel with the mobile industry, with their big fat phones that we put up with at the start of the industry. With augmented reality, since Steve Jobs allowed access to the camera in the iPhone, the industry has just boomed. Now we can have these experiences not just through smartphones but through eyewear, which is becoming more common. I believe in three to five years, it will grow towards maybe three t five percent of the population, and it will be quite a common thing in three to 10 years. By then, it will be representing a more fashion-conscious type of wearable device.

SE: I would agree with you, and I would go beyond that. In about the seven to 10 year mark, I think we’re going to see smart contacts, where AR is literally just going to be a part of how we get dressed in the morning.

SO: Absolutely. The device is just a stepping stone in a way.

SE: In the here and now, I actually see that AR is a usable technology. If I’m walking in the supermarket with my smartphone, isn’t it possible, because of the camera in my phone, that I can have it pointing to shelves of things and pop up, “This one is good for your diet,” “This other product is bad for your diet,” and give me that kind of contextual overlay that will help me make a smart shopping choice?

Is Apple going 3D?

SO: Absolutely. We’re jumping from a 2D world to a 3D world. Now our phones are becoming more intelligent by being able to sense the world. Also, at the moment, our cameras within our phones understand the world in 2D. They will begin to understand the world in 3D. That will give us incredible insights into how to shop, where to shop, and, I think, also we will start to have greater leverage of data sets that are currently dormant because we’re restrained in this 2D world. The next one to two years will be very exciting. Apple having bought PrimeSense, who supplied 3D depth sensing to Microsoft, could now make their own 3D depth sensing. There are other companies supplying 3D depth sensing cameras and software packages. That is a monumental step for us to have devices which, some call it, “wingman” to the world.

SE: I’m actually quite excited about Intel’s RealSense 3D chip that allows image recognition and facial recognition and is even able to detect moods. I would imagine that there’s an AR play in having that kind of technology in our smartphones, tablets and wearable devices. What’s your take?

SO: Absolutely. One of the famous acquisitions that occurred a couple of years ago was Hewlett-Packard buying a company called Autonomy. That was at around the time of the London Olympics, and a lot of people looked at Autonomy’s product called Aurasma for its print and augmented reality features, but actually HP bought Autonomy for its facial recognition for security technologies.

Whilst commercially a lot of people have seen this print to digital augmented reality experience from newspapers and magazines or packaging, facial recognition has been a very strong impetus for heavy investment. There are some commonalities in the programming but, by and large, in the vast expanse of the augmented reality world, they are different disciplines. Being able to understand people’s emotions or simply identity them has huge implications, from the line-ups at customs through to theme parks and rewarding people in-store for their frequency of visits.

SE: Even simple things like the camera in your phone recognising your friend while you are walking down the street, even though your vision may not be good enough to have picked them out.

SO: Well, never mind our vision — sometimes our memory.

SE: Yes.

SO: There’s Dunbar’s number which suggests we can reasonably sustain about 150 friendships. Some people say that varies between 100 and 250. Still, we come upon thousands of people throughout the year. When we go to conferences or schools or wherever, we’re asked to recall instantly many more than Dunbar’s number. Augmented reality has a very significant impact in being able to, I guess, resort to the cloud and overlay intelligence so that, when we look at a person’s face who might be outside our top 150 recalled faces, we don’t suffer the embarrassment of forgetting the name of someone who we really should know.

SE: Or someone we haven’t seen in 10 years and their appearance has changed a bit, and we just don’t recognise them, and they come up, “Hey, Scott, how are you doing,” and suddenly you’re like, “Okay, where do I know him or her from?” Boom, you’ve got the answer.

Great potential for facial recognition

SO: Absolutely. It’s a stepping stone, in a way, to becoming superhuman because what happens when, say, 10 or 20 percent of the population have this ability, have this device to see and recall faces straightaway? The other 80 percent or whatever it may be, will be missing out. What happens when there’s an expectation that not only should you remember people that you’ve been exposed to before, but that you should know the names and faces of friends of friends who you have never been exposed to? It’s absolutely a tool for enterprise and for personal lifestyle or engagement.

SE: It goes beyond that, too, in my opinion, because not only can you recognise faces but you can tie into all sorts of social media networks and get instant updates about what a person is up to and engage in a relevant conversation.

SO: Absolutely. It becomes a bridge to a new conversation around a common interest. It means that, when you’re in an event or in an education sphere, you can connect to people more meaningfully, more instantly. We have less wasted time, less beating around the bush.

SE: Yes. I see that the technology pieces are here now, but they still just need to be integrated into an application. You’re very much at the cutting edge of doing that kind of integration. What’s your prognosis for time frames?

SO: Well, a couple of years ago Google purchased a company called Viewdle for this very reason. Also about 12 months ago there was a lot of discussion around Google Glass and its ability to identify faces. We have a legal privacy situation to deal with, but as Marissa Mayer of Yahoo suggests, there is this new realm of privacy where, if function is compelling enough to overcome people’s hesitation about privacy, the new function will proceed and privacy will fall by the wayside. I think the ability to recognise faces provides a very compelling function.

I see our anxieties around privacy subsiding over the next — maybe two to four years. Just like in the early days of mobile when cameras were all built into Nokias, it took society a good one to three years to get comfortable with that, and with having their photos taken in social settings, from pubs to other public places. I feel the same scenario will happen with augmented reality eyewear.

SE: It’s a matter of timing. I would imagine also that there are many other consumer-oriented applications where AR can play a big role. What would you name as the top candidates for AR consumer-based applications?

SO: Well, I believe that human-to-human communication will be an outstanding opportunity. I believe that augmented reality has had many great use cases up to now across many different verticals — education, pharmaceutical, retail, through to military. Human-to-human communication, I believe, is what will help grow the technology as an option amongst consumers at an accelerated rate.

SE: Describe human-to-human communication with an AR boost. How would that work? What would a person do that’s different from what the do today?

The power of the hologram

SO: Well, one of the easiest ways to jump to a definition is recalling Princess Leia, “Star Wars,” 1977. That was more or less a projection from R2-D2 into a mist. That is possible, but that requires a little bit of a special physical setup. Instead of projecting into a mist, we can augment through a smartphone [camera view]. We can see a life-sized person, photo-real, and more or less mimic in a digital way face-to-face, eye-to-eye, whites-in-the-eye conversation. It’s a rather compelling way to experience, for want of a better term, holograms.

SE: If you were in California and I were in Sydney and we wanted to do this interview, we could literally feel like we’re sitting across the table.

SO: That kind of communication has been in an aspirational state for 40 years. In these next 12 months, we’ll move it into science fact.

SE: How is that different from telepresence technology which has been around in the corporate world for a long time now?

SO: Telepresence, which is being led ably by Cisco, has been largely a 2D video experience. With that we have video screens set up so we can be face-to-face but we don’t really have that direct whites-in-the-eye contact, and that is very important. Because when you have that whites-in-the-eye contact, neurotransmitters fire back and forward. Those neurotransmitters are responsible for trust, loyalty, persuasion and bonding, which obviously are very important in the business world and in personal relationships.

Stepping from the 2D, video type of experience into more of a hologram, eye-to-eye, life-sized, photo-real experience, we’re getting closer to those neurotransmitters firing. Now, if we were to take our eyes just two to three degrees away from looking at the whites in the eyes, we would lose that firing. So it’s rather critical that we get that kind of alignment and mimic it as close as possible in digital.

SE: It sounds to me like a very powerful marketing and advertising tool.

SO: Absolutely. It’s hard to imagine anything coming closer, and because we have a lot of access to video across mobile now, with clearing one billion smartphones in the world, not only developed countries but developing countries will have access to these kinds of technologies that are enabling these so-called hologram experiences. We can overcome massive distances using this hologram-like technology. So-called hologram or hologram-like. It’s not the labs’ and hard core scientists’ definition of a hologram. What we’re doing is augmenting space and time. For 99 percent of the population, when they see a digital human, they’ll call it a hologram. For communication purposes, we’re also using the word hologram.

Enhanced viewing of sporting events

SE: Well, turning to another subject. One of the other things that you mentioned was the use of augmented reality in sporting and entertainment. One of the ideas that I’ve come across there is that a viewer will be able to get all kinds of different angles of, say, a sporting event or a concert, maybe even being able to see from behind the stage or on the stage out into the audience or a bird’s eye view of what a player might see on the field. Is there much happening in this area?

SO: It is amazing to watch the development of innovation in video. Yes, there are prototypes being developed (Triggar, NextVR, Livelike), but I feel there is a long way to go in terms of what’s possible. User-generated content has been a high-growth area in video.

What I think is the next wave of growth is point-of-view video, which can be like an augmented or virtual experience. The point of view from whom — it might be from Jack Nicholson’s seat by the court, in his favourite basketball arena. It might be from the helmets of the NFL or NHL players. There’s a number of ways to capture and monetise brand-new experiences, which help the armchair or the in-stadium spectators become like fighter pilots with this enhanced vision of the activity that they’re passionate about.

Often, when we’re watching sport, we’re in this high-dopamine moment. We’re very excited. We’re very emotional. We’ve probably got our wallets and hearts open, and we’re willing to make micro-investments in our favourite players, in our favourite pastimes. This offers a tremendous opportunity to rights holders in the sports and entertainment world.

SE: Now I can see huge merchandising possibilities, possibilities for affiliated charities and causes, an ability to sell season tickets. All sorts of things come to mind.

SO: Absolutely.

SE: Turning to another area which is AR or VR or holograms in the enterprise, what would you pick as your top applications there?

Finance industry taking the lead

SO: In the last year, out of all industries that have been approaching me, the finance industry has led the charge, and I think that’s because of two reasons. One is finance brands are often known as very dry. Secondly, we’re not just in a finance economy. We’re in an attention and engagement economy. Out of all the verticals, I think that finance brands actually suffer the most for attention and engagement. I think there is a huge opportunity for finance brands to use augmented reality and virtual reality to explain their products, to build human-to-human relationships, to also combine augmented reality and virtual reality with artificial intelligence. I think there is a huge opportunity for other verticals as well. In the enterprise, I like the idea of how finance brands have been reaching out the most. Rather than other verticals that people refer to as leading technology charges, I think finance brands could be the leaders for the next year.

SE: Can you give just a very brief example of a finance brand that’s doing something cool with AR?

SO: One that many people, in Australia at least, have experienced is the CommBank App, where you have data sets that were previously locked in to a 2D screen with information about the latest house prices and ratings. These have been leveraged into a 3D augmented reality experience. You can point a camera, a smartphone, to certain homes and then, in front of them, you have intelligence of their most recent sale price or what their prospective sales price would be, some new information about the seller and so on. Again, a bit like the sports fan we were talking about earlier, it turned the prospective homebuyer into a super-enhanced, informed, fighter pilot. It rapidly, I guess, converted people not only to the bank’s application but induced those people new to the application to come back. It cut down the cost of customer acquisition and also delighted the people that downloaded the app.

SE: Well, one last question that I’d like to ask before we close off for today, and that has to do with devices that we use for augmented reality. Having a smartphone in our hand and scanning the environment gets tiring after a while. We have some glasses, specialised glasses like Google Glass or Laster SeeThru glasses, and others that are designed specifically for AR, but frankly a lot of the AR glasses that I’ve tried on are really big and kludgy, and they’re not comfortable to wear. What kind of timeframe do you think it will be until we have really wearable eyewear that has an AR overlay?

SO: There’s a couple of things to point out there. By and large, I think having something that can sit on your nose which is sub 100 grams is probably one to three years away. The weight of our sunglasses and reading glasses is between 20 to 30 grams, so our noses, our faces, can handle more or less up to 100 grams. Currently, the augmented reality eyewear is 200 to 300 grams. There are different compromises we can make to get it down to 100 grams, and then also there is a general improvement of technology over time.

I liken the progress to what’s happening in the music industry. Music went from records to cassettes, to CDs and DVDs, to MP3s, to downloading, to streaming. With augmented reality, we are moving from desktop, laptop, cameras, through to smartphones, through to eyewear, then through to contact lenses.

Augmented reality will enjoy a smaller form factor, I believe, in one to three years. Having said that, it’s interesting to watch the phenomenon of things like Beats, where for many years it was all about small form factor earphones, which ironically Apple were promoting. Since we know Apple has bought Beats, and Beats ironically led the charge for the more ostentatious, larger form factor. I think it’s not just about how large the eyewear is. It’s how well it’s presented as a fashionable item and possibly how it is led by influencers and, as with Beats earphones, seeded in music video clips.

SE: I think eyewear also needs to be comfortable. If it doesn’t feel good to wear, people will not keep it on their face for any period of time, even if the coolest person on the planet wears it. If it hurts, it’s going to come off.

SO: That’s the overarching principle, absolutely.

SE: Any last words before we close off, Scott?

SO: Well, it’s an exciting time to be alive. In one sense, augmented reality is allowing humans to be the most intelligent we’ve ever been. We sometimes hear of smartphones being a prosthetic for the brain, but, with augmented reality on top of that, it’s an exponential improvement. Further to this, I’m very excited about creating a human-to-human hologram-type interaction using augmented reality. I’m looking forward to delivering that within these 12 months. This will connect the planet and seven billion residents on the planet like never before, I believe. I cannot wait to share this with as many people as possible.

SE: Thank you so much, Scott. We’ll be talking a bit later once you’ve launched your hologram experience. Really appreciate your time today.

SO: Thank you for having me. It’s been a pleasure

About the author: Shara Evans is internationally acknowledged as a cutting edge technology futurist, commentator, strategy advisor, keynote speaker and thought leader, as well as the Founder and CEO of Market Clarity.