Unity Catalog

Overview

Unity Catalog is a catalogue for data and AI assets, open-sourced by Databricks in June 2024. Read that carefully when comparing: there is an open-source project of this name, and there is the Unity Catalog built into the Databricks platform. They share APIs and a name, but the open-source server is a much smaller feature set.

What sets it apart from an Iceberg-only catalogue is what it will hold. Alongside tables it catalogues volumes, which are non-tabular data such as JSON, text or Lance datasets, plus functions and models, so governance covers AI assets and not only warehouse tables.

Assets live in a three-level namespace of catalogue, schema and asset. Formats include Delta Lake natively, Iceberg and Hudi through UniForm, and Parquet, JSON and CSV.

Interoperability is its stated purpose. It is compatible with the Hive metastore API and serves the Iceberg REST catalogue API, so UniForm-enabled tables can be read by Iceberg clients. Documented engine integrations include Spark, Trino, DuckDB, Daft, StarRocks and PuppyGraph, plus AI clients such as LangChain and LlamaIndex.

Governance in the open-source server is basic: hierarchical privileges, external OAuth identity providers and credential vending through storage credentials. Row and column filters, attribute-based policies and audit logs belong to the Databricks service, not the open-source project.

Two practical notes. The open-source project is a sandbox project at LF AI & Data, the foundation's earliest maturity tier. And table maintenance, scalability limits and an official Helm chart are not published for the open-source server, which runs from a tarball or Docker Compose over PostgreSQL or MySQL.

Key features and capabilities

The same headings are used for every table & technical catalogues entry, so two tools can be read side by side.

What it catalogues

• Three-level namespace: catalogue, schema, asset
• Asset types: tables, volumes, functions and models
• Volumes hold non-tabular data such as JSON, text files or Lance datasets
• The Databricks service adds views and model or MCP services

Table formats

• Delta Lake natively, with Iceberg and Hudi through UniForm
• Parquet, JSON, CSV and others
• UniForm tables carry both Delta and Iceberg metadata
• Native Iceberg writes in the open-source server are not published

Engines that can use it

• Documented integrations: Spark, Trino, DuckDB, Daft, StarRocks, PuppyGraph, Kuzu, SpiceAI, XTable
• Spark integration needs Spark 3.5.3 and Delta Lake 3.2.1 or later
• AI clients: LangChain, LlamaIndex, OpenAI, Anthropic, CrewAI, AutoGen

Access control and auditing

• Hierarchical privileges such as use catalog, use schema and select
• External OAuth identity providers, plus a local user database
• Credential vending through storage credentials and external locations
• Row and column filters, attribute-based policies and audit logs are Databricks features, not open source

Interoperability

• Published OpenAPI specification, server and clients
• Compatible with the Hive metastore API
• Serves the Iceberg REST catalogue API for UniForm tables
• AWS Glue can federate to the Databricks service

Table maintenance and operations

• Table maintenance, compaction and snapshot expiry are not published for the open-source server
• Backing store is PostgreSQL or MySQL, with H2 in memory for testing
• Scalability limits and quotas are not published

How it runs

• Self-hosted from a tarball, or Docker Compose for a quick start
• No official Helm chart is published
• Managed inside Databricks on AWS, Azure and Google Cloud

Pricing

Open source

The open-source project is free under the Apache 2.0 licence. Databricks does not publish a separate price for its managed Unity Catalog; it is part of the platform, where billing is based on compute.

Vendor pricing page →

Demos and videos

• Open sourcing Unity Catalog Product demo · YouTube
• Getting started with Unity Catalog: a step-by-step demo Tutorial · YouTube

About Databricks / LF AI & Data Foundation

Unity Catalog was built by Databricks and open-sourced in June 2024 under the Apache 2.0 licence. It is hosted by the LF AI & Data Foundation as a sandbox project, the foundation's lowest maturity tier, and Databricks is the dominant contributor: the published roadmap is to port functionality from the Databricks service out to the open-source project in stages. The managed Unity Catalog inside Databricks remains a proprietary platform feature. Version 0.6.0 of the open-source server arrived in August 2026.

unitycatalog.io