“I’ve been a true believer in the metaverse for over thirty years, but I’m also deeply concerned about the power that metaverse platforms will have over the lives of consumers unless we implement thoughtful regulation.”
An interview with Louis Rosenberg, CEO and Chief Scientist at Unanimous AI
The term metaverse is everywhere. It has become the new buzzword in the tech world and in internet technology. The promise of a “metaverse” is being used by companies across entertainment, media, tech and gaming to lure developers and excite investors. But what exactly is it, why it matters, and how does it work? To answer these questions, we turned to metaverse pioneer Dr. Louis Rosenberg, founder of Immersion Corporation, Outland Research, and Unanimous AI.
Q: Can you please tell us a few words about you, your involvement with the metaverse, and your latest company Unanimous AI?
Louis: I have been involved in the metaverse for over 30 years, starting back in 1991 when I was researching virtual reality interfaces at Stanford and NASA. I was quickly convinced that virtual worlds would one day transform society, but I found the experience inside VR to be somewhat isolating, making my world smaller, not larger. What I really wanted was to take the power of VR and spread it all over the real world so we could experience both at the same time. This inspired me to propose the concept of augmented reality (AR) to the US Air Force. They gave me a research fellowship, which brought me to Air Force Research Laboratory (AFRL) in 1992 at Wright Patterson Air Force Base where I developed the Virtual Fixtures platform, the first functional augmented reality system that allowed users to interact with a single immersive world of real and virtual objects. The work involved testing VR and AR on human subjects, who would all tell me how captivating the technology was. This inspired me in 1993 to found the early virtual reality company Immersion Corporation which I ran for a decade. I then founded an early augmented reality company Outland Research in 2004. Currently I am the founder and CEO of Unanimous AI, an artificial intelligence company that amplifies group intelligence in shared environments. Along the way, I also wrote a dystopian graphic novel about the metaverse (UPGRADE) in 2008 which is a cautionary tale about the abuses that could happen when society adopts VR at a large scale. I’ve been a true believer in the metaverse for three decades, but I’m also deeply concerned about the power that metaverse platforms will have over the lives of consumers unless we implement thoughtful regulation.
Q: For most of us, the metaverse is still an unknown world. We have no idea how and according to what rules it will work. So, what are people expected to do on the metaverse (especially when it comes to opportunities, threats and regulation of our personal data)?
Louis: I like to describe the word “Metaverse” as the global transition from flat media viewed from the outside in the third person to immersive media experienced from the inside in the first person. In other words, the metaverse means that users achieve a sense of “presence” with respect to content, information, and other users – something that we don’t achieve with flat media. This will change our relationship with computing at all levels, enabling far more realistic and captivating experiences for everything from shopping and socializing to business, education, and entertainment.
To appreciate the massive impact the metaverse will have on society, it’s important to realize there will be two branches – the virtual metaverse and the augmented metaverse:
The virtual metaverse will involve VR headsets and will be a fully simulated world, much like the marketing videos that are currently promoted by Meta and other platform providers. Users will interact as avatars that look cartoonish today but will get better and better until they achieve photorealistic quality. While deeply immersive, this world will be used mostly for socializing and casual forms of entertainment from gaming and watching sports to shopping with friends. The virtual metaverse will be quite popular but will only be used a few hours a day by most people. That’s because wearing bulky headsets is not pleasant for extended use.
The augmented metaverse will involve lightweight glasses that overlay virtual content on the real world. This will transform our lives, bringing magical virtual content into all aspects of our day from the moment we wake to the moment we go to sleep. AR glasses will be sold by Apple, Google, Samsung, Sony, Meta, Snap and other major manufacturers and within 10 years’ time will replace mobile phones as the most common method of interacting with digital content. Instead of information being restricted to a small screen in your hand, it will be splashed all over the real world, embellishing all of your experiences and interactions. By the early 2030’s we will look back at a time when people walked down the street gazing down at their phone and laugh. To help convey what our lives will be like in the near future, I wrote a popular narrative entitled Metaverse 2030 that is available here.
At the same time, the metaverse will create significant new problems related to privacy and predatory business practices that are very concerning and require aggressive regulation. That’s because in the metaverse, platform providers will be able to track everything you do – where you go, who you are with, what direction you’re looking, how long your gaze lingers, whether you slow down while walking past something to pay extra attention or speed up and rush past. They will also track your facial expressions and your vocal inflections and use that information to analyze your emotional response to all the interactions you have. Multiple companies are even working on technology to track your vital signs (blood pressure, heart rate, respiration rate) using sensors in earbuds to give even deeper tracking of emotions. This level of tracking and monitoring goes beyond anything we have seen in human history and has great potential to be abused by the large corporations that will control metaverse platforms.
Q: The metaverse is a virtual world focused on social experiences. But the metaverse differs from current social networks. Today’s social media allow us to target advertising and news. How will the metaverse impact advertising and news?
Louis: In today’s social media we are all targeted with ads, news, and other content based on the personal data and information collected about us. Because the information tracked will be far more extensive in the metaverse, the targeting will be far more precise. That’s disturbing, but it’s not the real danger. The danger comes from the type of advertisements and other content users will be targeted with.
As I mentioned above, the metaverse is a transition from flat media experienced in the third person to immersive media experienced in the first person. This means that advertisements will no longer be flat images and videos but will transform into interactive experiences that are naturally integrated into your virtual or augmented world. These experiences will include virtual product placements where products, services, and other promotional content will be added to your world as you go about your day. Unless regulated, you might not be able to tell the difference between natural experiences in virtual and augmented worlds and promotional experiences that were injected into your life on behalf of a paying advertiser. This is dangerous.
And it gets worse. That’s because the promotional vehicles used in the metaverse will not just be static product placements, but will also involve AI-controlled avatars (virtual people) that you might pass on the street (virtual or augmented) who are wearing particular brands or holding shopping bags of particular stores or drinking particular brands of cola. You may think those people are just other users in the metaverse that you came upon by chance, when really, they were injected into your world specifically to target you with the objective of influencing your views of products, services, and ideas. And unless regulated, you might not be able to tell the difference between authentic users and AI-driven virtual people injected into your life on behalf of a paying advertiser.
And it still gets worse – that’s because these virtual people (or Veeple as I call them) will also engage us in promotional conversation on behalf of sponsors. In fact, this will likely become a primary means of advertising in the metaverse and not just for products and services, but for political messaging, propaganda and misinformation. These “agenda-driven AI agents” will be programmed to target you based on the detailed history of your actions and interactions inside the metaverse, drawing you into conversations in ways that will subtly influence your personal views and perspectives. Unless conversational advertising is regulated in the metaverse, users may not be able to tell the difference between a promotional conversation that was initiated on behalf of a paying advertiser and an authentic conversation that happened by chance with another user in the metaverse.
Q: Media studies have traditionally focused on producing, distributing, or consuming certain content. The rise of virtual and augmented reality provides new opportunities as consumers now seek more immersive experiences. Since journalism is not just about how to tell a story but what the business model looks like, how can we future-proof newsrooms for the metaverse?
Louis: The metaverse will not just change advertising, it will also change journalism and other forms of mass media. The transition will be profound – instead of reading or listening to content as a third person observer, journalists can bring their viewers into an experience and allow them to get a first-person view. This can be very powerful. In fact, one of the most powerful journalistic efforts that I ever experienced in the metaverse was a piece of undercover reporting created by documentary film-makers about the abusive practices of the factory farming industry. They created a virtual reality experience filmed in 3D with full 360-degree view, bringing viewers into a factory farm so they can see and hear the horror that animals experience from the inside. I’m not sure you could achieve a similarly powerful experience with flat video. I believe this will be used extensively in warzones and other environments where viewers could get much deeper information in the first person. Of course, similar tactics could be used for propaganda and misinformation, so the deeply moving power of first person reporting needs to be accompanied by the highest journalistic standards.
Q: From your experience, do you believe there is a true financial incentive for media organizations to invest in the metaverse? Especially, if we think that traditional revenue models (such as ads and subscriptions) have proved inadequate for previous immersive news content. Do you see an opportunity for media organizations to monetize on the metaverse?
Media organizations are already investing in the Metaverse and will ramp up considerably in the coming years. That’s because immersive content will replace flat content as the standard method by which we consume information. Of course, this requires a large installed base of hardware to display the content which does not yet exist. That said, I predict a very large installed base of hardware will soon emerge. I’m not talking about the bulky VR goggles of today, but stylish and light weight augmented reality glasses. These glasses will be launched in 2024 or 2025 by Apple, Google, Meta, Snap and others and will quickly transform the mobile media space from flat screens held in your hand to AR eyewear worn naturally.
Media organizations should expect rapid adoption of AR glasses once this hardware is released, because without the eyewear users will be missing out on content. They will feel pressured to buy the glasses. It will follow a very similar adoption curve as we saw when the iPhone was launched in 2007 and quickly replaced the flip-phone. Nobody knew they needed a smartphone back then and they might not have even wanted one, but within five or six years it became standard equipment because not having a smartphone meant missing out on content. This will happen with AR glasses. By the early 2030’s most consumers in industrialized nations will feel like they need AR glasses. It will quickly become a massive installed base and will drive immersive content for all media organizations.
Q: In a CGI (Computer Generated Imagery) metaverse, news content will be a representation of the physical world, not a digital capture of it, such as digital video. For instance, the BBC’s VR experience Easter Rising has made some strong AD/design choices. How objective, honest and transparent can that be?
Louis: The use of virtual studios to create completely fabricated content rather than recorded content will become a standard practice in the coming years. That’s because CGI content will soon be cheaper, faster, and easier to create than to film on location in many situations. This will drive a real need for increased journalistic integrity among media organizations. My personal view is that it should be strictly regulated – news organizations should be required to inform viewers when content is created using CGI rather than filmed on location. I say that because the quality will be so good, it will not be apparent to viewers without disclosure.
In addition, using CGI to create background scenes and other contextual content, we are getting close to the point where CGI can be used to replicate people with photorealistic accuracy. This creates an even larger danger as news organizations could easily interview a CGI representation of a political figure rather than the real person. This is often referred to as creating a “digital twin” and is becoming very cheap and easy to do. This too needs to be aggressively regulated. Fringe media organizations (and fraudsters) could easily create an “evil twin” of notable figures and use those CGI creates to spread misinformation and disinformation of the most dangerous kind.
Q: Occupying the border space between information and entertainment, journalism raises important questions about freedom of expression in virtual worlds. What are the threats and what do we need to focus on?
In my responses above, I express numerous times that we need to regulate metaverse platforms to protect consumers from being intrusively tracked, manipulated, and deceived by virtual content that looks so real it can’t be distinguished from genuine authentic experiences. That said, whenever someone calls for regulation, it’s important to also explore the implications to freedom of expression.
My view is that regulation should not focus on restricting expression in virtual and augmented worlds but should instead focus on transparency of metaverse platforms. I believe that content creators should be required to disclose when a piece of information is virtual rather than authentic so that consumers are not fooled into thinking something is genuine when it is CGI. I also believe that platform providers should be required to disclose when a piece of virtual content is injected into a user’s virtual or augmented world to specifically target that user on behalf of a paying advertiser. That’s because the user may otherwise assume they came across the content in a natural and serendipitous way and have no idea that a third party is actively manipulating their lived experiences. This type of transparency is critical to protect users from being influenced without their knowledge in virtual and augmented worlds.
About Louis Rosenberg
Louis Rosenberg, PhD is a technology pioneer in the fields of VR, AR, and AI. In the early 1990’s he developed the first functional augmented reality system for Air Force Research Laboratory (the Virtual Fixtures platform). He then founded the early VR company Immersion Corporation (1993) and the early AR company Outland Research (2004). He is currently CEO of Unanimous AI, a company that amplifies human intelligence in shared environments. He earned his PhD from Stanford University, was a professor at California State University (Cal Poly) and has been awarded over 300 patents worldwide for VR, AR, and AI technologies.
You can stay connected with Louis via Medium or follow him on LinkedIn and Twitter.
About Unanimous AI
Unanimous AI amplifies the intelligence of networked human groups. Using patented AI technology modeled on the biological principle of Swarm Intelligence, their award-winning Swarm platform (SaaS) empowers business teams to generate significantly more accurate forecasts, assessments, prioritizations and decisions. Their Swarm Insight technology amplifies the intelligence of consumer groups, generating AI-optimized insights that significantly outperform traditional methods.
