Unity Catalog Open Sourced by Databricks

Databricks Announces Open Sourcing of Unity Catalog for Data and AI Governance

Databricks, the Data and AI company, has announced the open-sourcing of Unity Catalog, the industry’s only unified solution for data and artificial intelligence (AI) governance across clouds, data formats, and data platforms. This initiative underscores Databricks’ commitment to open ecosystems, providing customers with the flexibility and control they need without vendor lock-in. The announcement marks a new era for open catalog standards for data and AI, with support from major partners such as Amazon Web Services (AWS), Google Cloud, Microsoft, NVIDIA, Salesforce, and others. Unity Catalog Open Sourced by Databricks.

Thank you for reading this post, don't forget to subscribe!

Key Features of Unity Catalog OSS

Interoperability: Unity Catalog OSS offers a universal interface supporting any data format and compute engine. It can read tables with Delta Lake, Apache Iceberg™, and Apache Hudi™ clients via Delta Lake UniForm, and supports the Iceberg REST Catalog and Hive Metastore (HMS) interface standards. It is interoperable with all major cloud platforms, compute engines, and data and AI platforms.

Unified Governance: Unity Catalog OSS enables unified governance across tabular data, non-tabular data, and AI assets such as ML models and generative AI tools, simplifying management, discovery, and development at scale.

Openness: With open APIs and an Apache 2.0 licensed open source server, Unity Catalog OSS maximizes flexibility and customer choice by enabling broad interoperability across various engines, tools, and platforms.

Industry and Partner Support

Unity Catalog OSS is the industry’s only universal catalog for data and AI. Since its introduction in 2021, Unity Catalog has helped over 10,000 organizations break down silos created by multiple single-purpose solutions.

Customer Testimonials:

AT&T: “With Unity Catalog’s open sourcing, we are encouraged by Databricks’ step to make lakehouse governance and metadata management possible through open standards,” said Matt Dugan, VP Data Platforms, AT&T.
Nasdaq: “Databricks’ decision to open source Unity Catalog provides a solution that helps eliminate data silos,” said Lenny Rosenfeld, VP, Capital Access Platforms, Nasdaq.
Rivian: “We are excited about Databricks open sourcing Unity Catalog and releasing Open APIs to bring interoperability across our data landscape without any concerns of vendor lock-in,” said Jason Shiverick, Director of AI Platforms, Rivian.

Supporting Cloud Partners:

AWS: “AWS welcomes Databricks’ move to open source Unity Catalog,” said Chris Grusz, Managing Director of Technology Partnerships, AWS.
Google Cloud: “Databricks’ strategy to open up the Unity Catalog standard for data and AI aligns very well with our strategy,” said Ritika Suri, Director, Data and AI Technology Partnerships, Google Cloud.
Microsoft: “We believe truly open standards with broad industry participation are in customers’ best interests,” said Jessica Hawk, CVP, Data, AI, Digital Applications, Microsoft.
Salesforce: “We are excited to have Databricks as a member of our Zero Copy Partner Network and look forward to joint innovations with the new open Unity Catalog,” said Ravi Loganathan, EVP, Salesforce.

Supporting Data and AI Partners:

Confluent: “We’re excited to see Databricks make a significant contribution to an open data ecosystem with Unity Catalog becoming open sourced,” said Shaun Clowes, Chief Product Officer, Confluent.
dbt Labs: “We are thrilled to announce our support for Unity Catalog OSS and the open APIs,” said Mark Porter, CTO, dbt Labs.
DuckDB Labs: “We are thrilled to partner with Databricks on Delta Kernel and the Unity Catalog open standard for data and AI,” said Hannes Mühleisen, CEO, DuckDB Labs.

The Future of Data and AI Governance

With the open-sourcing of Unity Catalog, Databricks continues to lead in data and AI governance, fostering an ecosystem of interoperable tools, universal support for data and AI assets, and built-in security. Unity Catalog OSS will be available at the Data + AI Summit, furthering Databricks’ mission to empower organizations with the tools needed for modern data and AI applications.