New Technology Risks

September 27, 2024in Generative AI

Organizations have always needed to manage the risks that come with adopting new technologies, and implementing artificial intelligence (AI) is no different.

Many of the risks associated with AI are similar to those encountered with any new technology: poor alignment with business goals, insufficient skills to support the initiatives, and a lack of organizational buy-in.

To address these challenges, executives should rely on best practices that have guided the successful adoption of other technologies, according to management consultants and AI experts. When it comes to AI, this includes:

Identifying where AI can help achieve organizational objectives.
Developing strategies to ensure the necessary expertise is available to support AI programs.
Implementing robust change management policies to facilitate smooth and efficient enterprise adoption.

However, AI presents unique risks that executives must recognize and address proactively.

Below are 15 areas of risk that organizations may encounter as they implement and use AI technologies:

Lack of Employee Trust
AI adoption can falter if employees don’t trust it. According to KPMG and the University of Queensland, 61% of respondents in a 2023 survey expressed ambivalence or unwillingness to trust AI. Similarly, a 2024 Salesforce survey found that 56% of AI users struggle to get what they need from AI, and 54% don’t trust the data used to train AI systems. Without trust, AI implementations are likely to fail.
Unintentional Biases
AI systems can produce biased results if the data or algorithms used are flawed. This is not merely theoretical; real-world examples, such as predictive policing algorithms that disproportionately target minority communities, underscore the potential for AI to perpetuate harmful biases.
Amplification of Biases and Errors
While human errors are typically limited in scope, AI can magnify these errors exponentially due to the vast scale at which it operates. A single mistake by an AI system processing millions of transactions can have far-reaching consequences.
AI Delusions
Many AI systems are probabilistic, meaning they produce the most likely response rather than a deterministic outcome. This can lead to inaccuracies, often referred to as AI hallucinations. Users must approach AI outputs with caution, as the technology is not infallible.
Unexplainable Results
The lack of explainability in AI systems, especially in complex and continuously learning models, can damage trust and hinder adoption. While achieving explainable AI is crucial, it also presents challenges, such as potentially reducing accuracy or exposing proprietary algorithms to security risks.
Unintended Consequences
AI can have unforeseen consequences that were not anticipated by enterprise leaders. This risk is significant enough to have been highlighted by global leaders, such as the U.N. Secretary-General, who warned of AI’s potential to exacerbate inequality and harm societal well-being.
Ethical and Legal Dilemmas
AI can create ethical challenges, such as privacy violations or the misuse of copyrighted material. As generative AI becomes more widespread, legal disputes, like the lawsuit against OpenAI for unauthorized use of copyrighted content, highlight the need for careful consideration of ethical and legal implications.
Loss of Enterprise Control
The rise of generative AI has fueled the growth of shadow IT, with employees using unauthorized AI tools at work. This lack of oversight can lead to security risks and noncompliance with organizational policies.
Unsettled Liability Issues
Legal accountability for AI-generated outcomes is still unclear. For example, if AI produces faulty code that causes harm, it remains uncertain who or what is to blame, leaving organizations in a precarious legal position.
Compliance with Future Regulations
Governments worldwide are considering new regulations for AI. Organizations may need to adjust or even curtail their AI initiatives to comply with forthcoming laws, adding a layer of complexity to AI adoption.
Erosion of Key Skills
The reliance on AI could lead to the erosion of essential skills within the workforce. For instance, just as automation in aviation has raised concerns about pilots losing basic flying skills, AI could diminish other critical human competencies.
Societal Unrest
Fears of job displacement due to AI are growing, with many professionals worried about being replaced by automation. This anxiety could lead to societal unrest and demands for new ways of working and job retraining.
Poor Training Data and Lack of Monitoring
AI systems need high-quality training data and continuous monitoring to function correctly. Failures in these areas can lead to disastrous outcomes, as demonstrated by Microsoft’s Tay bot, which was quickly corrupted by malicious users.
AI-Driven Cybersecurity Threats
Hackers are using AI to create more sophisticated cyberattacks. AI enables even inexperienced attackers to develop effective malicious code, increasing the threat landscape.
Reputational Damage
Poor decisions around AI use can harm an organization’s reputation. For example, Vanderbilt University faced backlash for using AI-generated content in a sensitive communication, highlighting how missteps in AI use can lead to public criticism and reputational harm.

Managing AI Risks

While the risks associated with AI cannot be entirely eliminated, they can be managed. Organizations must first recognize and understand these risks and then implement policies to mitigate them. This includes ensuring high-quality data for AI training, testing for biases, and continuous monitoring of AI systems to catch unintended consequences.

Ethical frameworks are also crucial to ensure AI systems produce fair, transparent, and unbiased results. Involving the board and C-suite in AI governance is essential, as managing AI risk is not just an IT issue but a broader organizational challenge.