The Evolution of Data and AI: A New Frontier

Once considered the new oil, data is no longer the sole focus of business transformation. The emergence of artificial intelligence (AI) has shifted the conversation toward compute power as the fundamental resource driving innovation. Sequoia Capital emphasizes AI’s transformative impact, stating:

“The fields that generative AI addresses — knowledge work and creative work — comprise billions of workers. [If] Generative AI can make these workers at least 10% more efficient and/or creative: they become not only faster and more efficient, but more capable than before. Therefore, Generative AI has the potential to generate trillions of dollars of economic value.”
— Sonya Huang, Pat Grady, Generative AI: A Creative New World

Whether this vision materializes remains to be seen, but the comparisons to past technological revolutions—such as the internet and mobile computing—are hard to ignore.

The Role of AI in Knowledge Work and Data Analytics

Data analytics has seen unprecedented growth, prompting businesses to reassess its value. With a projected compound annual growth rate (CAGR) of 27.3% through 2030, the market’s rapid expansion has led to a push for better tools and standards. Data analytics has evolved from a discipline primarily rooted in physics, mathematics, accounting, and economics to a more structured academic field.

This shift is evident in the surge of data science degrees. The National Center for Education Statistics reported a 968% increase in data science bachelor’s degrees awarded from 2020 to 2022, highlighting growing demand.

Modern Data Analytics: From ETL to ELTL

Traditional extract-transform-load (ETL) processes have struggled to keep up with modern analytical demands. The introduction of extract-load-transform (ELT) architectures, championed by tools like dbt, empowered data teams by shifting transformation ownership from data architects to analysts. However, this speed came at a cost: fragmented data governance and inconsistencies in reporting.

A more refined approach—ELTL (extract-load-transform-load)—adds a critical governance step, ensuring structured data models before visualization and reporting.

Data Governance, the Semantic Layer, and AI

Unstructured data delivery has led to discrepancies across dashboards and reports. To address this, solutions such as dbt’s semantic layer and SQLMesh provide standardized testing, metadata management, and auditing mechanisms. AI plays a crucial role here, offering:

• Enhanced data modeling: AI-driven discovery of relationships between data points.
• Improved accuracy: Retrieval-augmented generation (RAG) techniques enrich AI models with metadata.
• Better governance: Guardrails for AI-generated insights, ensuring consistency.

AI-Driven Precision Analytics

AI is set to revolutionize data querying by replacing cumbersome dashboards with precise, real-time insights. Borrowing from the concept of precision medicine, “precision analytics” ensures that businesses receive tailored data transformations instantly. This shift reduces reliance on static filters and rigid dashboards, allowing AI to dynamically generate and refine queries.

The Rise of Data Objects

AI-generated SQL queries—what we term “data objects”—are distinct from traditional dashboards in that they are:

• Easily accessible and comparable.
• Defined by type: metric (1 row, 1 column), record (1 row, multiple columns), or dataset (multiple rows and columns).
• More flexible in adapting to evolving business needs.

AI Solutions: Retrieving and Querying Data

AI-powered data analytics falls into two broad categories:

1. Retrieving data from existing reports – AI-driven search across analytical reports can surface insights, but conflicting data across reports remains a challenge.
2. Querying raw data (text-to-SQL) – Advances in text-to-SQL technology enable AI to query databases directly, a capability now viable for enterprise use.

Building Trust in AI-Driven Analytics

AI-generated analytics must be trustworthy, requiring:

• Robust data governance – Well-documented schemas and business glossaries enable AI agents to function effectively.
• Human oversight – “Human-in-the-loop” workflows ensure accuracy and reliability.
• Guardrails – AI-driven workflows must incorporate validation steps like primary key checks and join definitions to prevent inaccuracies.

The Future of AI in Data Analytics

AI will not replace data teams but will enhance their efficiency by reducing bottlenecks and accelerating insights. Organizations that integrate AI with strong data governance will achieve a competitive edge, enabling faster, more accurate, and more relevant analytics.

The shift toward AI-driven data analytics is inevitable. The question is: will organizations embrace the change and refine their strategies, or will they be left behind in the wake of innovation?