Databricks Open Sources Unity Catalog

Databricks Open Sources Unity Catalog

Unity Catalog OSS offers a universal interface that supports any data format and compute engine, including the ability to read tables.

Databricks, the Data and AI company, announced that it is open sourcing Unity Catalog, the industry’s only unified solution for data and artificial intelligence (AI) governance across clouds, data formats and data platforms.

“Our customers love Unity Catalog. It lets them manage all their data objects — tabular data, unstructured data, and AI and ML assets — in a single source of truth within the Databricks Data Intelligence Platform, versus gluing together multiple single-purpose solutions,” said Ali Ghodsi, Co-founder and CEO at Databricks. “Our platform is the only major data platform in the industry where all data is in an open format by default — now, metadata and governance are open as well, giving enterprises the governance solution they need in today’s data and AI landscape. We’re excited to open source Unity Catalog and release the code. We’ll continue to evolve the open standard in close collaboration with our partners.”

This initiative builds on Databricks’ commitment to open ecosystems, ensuring customers have the flexibility and control they need without vendor lock-in. Databricks is ushering in a new era for open catalog standards for data and AI with support from Amazon Web Services (AWS), Google Cloud, Microsoft, NVIDIA, Salesforce, and more.

Unity Catalog OSS offers a universal interface that supports any data format and compute engine, including the ability to read tables with Delta Lake, Apache Iceberg, and Apache Hudi clients via Delta Lake UniForm. It also supports the Iceberg REST Catalog and Hive Metastore (HMS) interface standards.

Also Read: Qlik Leverages Databricks AI Capabilities

Additionally, Unity Catalog OSS provides for unified governance across tabular, non-tabular data, and AI assets, such as machine learning (ML) models and generative AI tools, letting organisations simplify management at scale.

Unity Catalog OSS is the industry’s only universal catalogue for data and AI. It offers a universal interface that supports any data format and compute engine, including the ability to read tables with Delta Lake, Apache Iceberg, and Apache Hudi clients via Delta Lake UniForm. It also supports the Iceberg REST Catalog and Hive Metastore (HMS) interface standards.

With its open APIs and Apache 2.0 licensed open source server, Unity Catalog OSS maximises flexibility and customer choice by enabling broad interoperability across various engines, tools, and platforms.