Gen AI Depends on Good Data

Accelerate Your Generative AI Journey: A Call to Action for Data Leaders

Generative AI is generating immense excitement across organizations, with boards of directors conducting educational workshops and senior management teams brainstorming potential use cases. They need to keep in mind, Gen AI Depends on Good Data. Individuals and departments are already experimenting with the technology to enhance productivity and effectiveness. There needs to be as much effort into data quality as to the technology.

Thank you for reading this post, don't forget to subscribe!

The critical work required for generative AI success falls to chief data officers (CDOs), data engineers, and knowledge curators. Unfortunately, many have yet to begin the necessary preparations. A survey in late 2023 of 334 CDOs and data leaders, sponsored by Amazon Web Services and the MIT Chief Data Officer/Information Quality Symposium, coupled with interviews, reveals a gap between enthusiasm and readiness. While there’s a shared excitement about generative AI, much work remains to get organizations ready for it.

The Current State of Data Preparedness

Most companies have yet to develop new data strategies or manage their data to effectively leverage generative AI. This insight outlines the survey results and suggests next steps for data readiness.

Maximizing Value with Generative AI

Historically, AI has worked with structured data like numbers in rows and columns. Generative AI, however, utilizes unstructured data—text, images, and video—to generate new content. This technology offers both assistance and competition for human content creators. Survey findings show that 80% of data leaders believe generative AI will transform their business environment, and 62% plan to increase spending on it. Yet, many are not yet realizing substantial economic value from generative AI.

Only 6% of respondents have a generative AI application in production deployment. A significant 16% have banned employee use, though this is decreasing as companies address data privacy issues with enterprise versions of generative AI models.

Focus on Core Business Areas

Experiments with generative AI should target core business areas. Universal Music, for instance, is aggressively experimenting with generative AI for R&D, exploring how it can create music, write lyrics, and imitate artists’ voices while protecting intellectual property rights.

Gen AI Depends on Good Data

For generative AI to be truly valuable, organizations need to customize vendors’ models with their own data and prepare their data for integration. Generative AI relies on well-curated data to ensure accuracy, recency, uniqueness, and other quality attributes. Poor-quality data yields poor-quality AI responses.

Data leaders in our survey cited data quality as the greatest challenge to realizing generative AI’s potential, with 46% highlighting this issue.

Jeff McMillan, Chief Data, Analytics, and Innovation Officer at Morgan Stanley Wealth Management, emphasizes the importance of high-quality training content and the need to address disparate data sources for successful generative AI implementation.

Current Efforts and Challenges

Most data leaders have not yet made significant changes to their data strategies. While 93% agree that a data strategy is critical for generative AI, 57% have made no changes, and only 11% strongly agree their organizations have the right data foundation.

Organizations making progress are focusing on specific tasks like data integration, cleaning datasets, surveying data, and curating documents for domain-specific AI models. Walid Mehanna, Group Chief Data and AI Officer at Merck Group, and Raj Nimmagadda, Chief Data Officer for R&D at Sanofi, stress the importance of robust data foundations, governance, and standards for generative AI success.

Focus on High-Value Data Domains

Given the monumental effort required to curate, clean, and integrate all unstructured data for generative AI, organizations should focus on specific data domains where they plan to implement the technology. The most common business areas prioritizing generative AI development include customer operations, software engineering, marketing and sales, and R&D.

The Time to Start is Now

While other important data projects exist, including improving transaction data and supporting traditional analytics, the preparation for generative AI should not be delayed. Despite some slow pivoting from structured to unstructured data management, and competition among CDOs, CIOs, CTOs, and chief digital officers for leadership in generative AI, the consensus is clear: generative AI is a transformative capability. Preparing a large organization’s data for AI could take several years, and the time to start is now.