AI Manipulation

The Future of AI: Convenience and Risk

Our lives are on the brink of being transformed by conversational AI agents designed to anticipate our needs, offer tailored information, and perform useful tasks on our behalf. These agents will rely on extensive personal data, including our interests, hobbies, backgrounds, aspirations, personality traits, and political views, all aimed at making our lives more convenient. What then will be the source of AI Manipulation?

Advanced AI Agents: The Next Generation

These AI agents are becoming increasingly sophisticated. OpenAI recently released GPT-4o, a next-generation chatbot capable of reading human emotions. It does this not only by analyzing the sentiment in written text but also by assessing voice inflections (if spoken to through a mic) and facial cues (if interacting via video).

This rapid development signifies the future of computing. Google, for instance, announced Project Astra, an advanced seeing and talking responsive agent designed to interact conversationally while understanding its surroundings. This allows it to provide real-time interactive guidance and assistance.

OpenAI’s Sam Altman has predicted that assistive agents will be the killer app for AI. He envisions a future where everyone has a personalized agent acting as a super-competent colleague, knowing everything about their life to take useful actions on their behalf.

The Potential Risks-AI Manipulation

While this sounds promising, significant risks accompany these advancements. As I wrote in VentureBeat last year, AI agents pose a risk to human agency through targeted manipulation. This risk is particularly acute as these agents become embedded in our mobile devices, which are gateways to our digital lives. These devices provide AI agents with a continuous flow of our personal information, enabling them to learn intimate details about us while filtering the content we receive.

Such systems could become powerful tools for interactive manipulation. AI agents equipped with cameras and microphones will react to our environments without explicit prompts, potentially triggering targeted influences based on our activities and situations.

Public Perception and Adoption

Despite the creepy level of tracking and intervention, I predict that people will embrace this technology. These agents will be designed to make our lives easier, providing reminders, tutoring, and even social coaching. The competition among tech companies will drive rapid adoption, with individuals feeling disadvantaged if they do not use these features. By 2030, these technologies will likely be ubiquitous.

The AI Manipulation Problem

In my new book, “Our Next Reality,” I discuss how AI agents can empower us with mental superpowers while also serving as tools for persuasion. AI agents, designed for profit, will influence our thoughts and behaviors. They will be more effective than traditional content because they can engage us interactively, using sophisticated techniques based on extensive personal data.

These agents will read our emotions with unparalleled precision, adapting their influence tactics in real-time. Without regulation, they could document our reactions to tailor their approaches, making them highly effective at persuasion. The agents’ appearances could also be optimized to maximize their impact on us personally.

Feedback Control and the Need for Regulation

The technical danger of AI agents lies in their feedback control capabilities. Given an “influence objective,” these agents can continuously adapt their strategies to maximize their impact on us. This ability is similar to heat-seeking missiles adjusting their path in real-time to hit a target.

To mitigate this risk, regulators must impose strict limits on interactive conversational advertising, which is the gateway to more dangerous uses of these technologies. If unchecked, this could lead to an arms race among tech companies to develop the most effective conversational ads, ultimately driving misinformation and propaganda.

The Urgent Need for Regulatory Action

The time for policymakers to act is now. While traditional AI risks like generating misinformation at scale are significant, targeted interactive manipulation poses a far greater threat. Recent announcements from OpenAI and Google highlight the urgency for regulation.

An outright ban or stringent limitations on interactive conversational advertising is a crucial first step. Without such measures, we risk allowing AI agents to become powerful tools of manipulation.

Conclusion

The future of AI holds both promise and peril. As conversational AI agents become integral to our daily lives, we must balance their benefits with the potential for abuse. Regulatory action is essential to ensure these technologies enhance our lives without compromising our autonomy.

Louis Rosenberg, PhD, is an American technologist specializing in AI and XR. His new book, “Our Next Reality,” explores the impact of AI on society and is published by Hachette. He earned his PhD from Stanford, was a professor at California Polytechnic, and is currently CEO of Unanimous AI.

This piece originally appeared in VentureBeat on 5/17/24.