Databricks Profiled

A lakehouse platform built on open formats (Delta Lake, Apache Iceberg) and Apache Spark, with unified governance through Unity Catalog.

At a glance

Databricks created the lakehouse approach: warehouse-grade reliability and SQL performance on open table formats in cloud object storage. It began as a managed Apache Spark service and now covers ingestion, pipelines, SQL warehousing, BI, ML and GenAI, all governed by Unity Catalog. It runs on all three major clouds.

Where it fits

  • Strongest when: you want open formats, multi-cloud, heavy data engineering or ML/AI workloads, and code-first teams.
  • Consider carefully when: the team wants a no-ops SaaS experience with minimal tuning. Serverless options narrow this gap, but check that they're available in your region.

Key concepts

  • Account → Workspace → Unity Catalog metastore → catalog.schema.table
  • Classic vs serverless compute: who owns the VMs and how you're billed
  • Medallion architecture: bronze, silver and gold layers, a common modelling pattern
  • Open by default: Delta Lake, Delta Sharing, MLflow and Unity Catalog are all open source

Products (12)

BI & Analytics

Built-in dashboards plus Genie, a natural-language interface for business users to query governed data.

Data Apps & Development

Hosts data and AI applications (e.g. Streamlit, Dash, Gradio) directly on the Databricks platform.

Data Integration (ETL/ELT)

Databricks' data engineering suite for ingestion, declarative pipelines and orchestration.

Data Science & ML

A hosted version of open-source MLflow for experiment tracking, model registry (in Unity Catalog) and GenAI evaluation.

Databricks' AI tooling for building, serving and monitoring ML models and GenAI agents.

Data Sharing & Marketplace

A privacy-safe environment for multiple parties to collaborate on joint data without exposing raw data.

An open protocol for securely sharing live data with any computing platform, not only Databricks.

Data Warehouse

SQL warehouses (serverless, pro and classic) for BI and analytical workloads on the lakehouse.

Governance & Catalog

Unified governance for data and AI assets (tables, volumes, models, functions), with access control, lineage and auditing across workspaces.

Lakehouse

An open-source table format that adds ACID transactions, time travel and schema enforcement to Parquet on object storage.

Operational Database

Lakebase

Preview

A managed, Postgres-compatible operational database integrated with the lakehouse and Unity Catalog.

Upcoming events

From the blog

Capability snapshot

24 native · 1 partial

CapabilitySupportNotes
Platform & Deployment
Cloud availabilityNativeAWS, Azure (Azure Databricks, a first-party Azure service) and Google Cloud.
Deployment modelNativeControl plane run by Databricks; classic compute runs in the customer's cloud account, and serverless compute runs in Databricks' account.
Pricing modelNativeDBU-based.
Storage & Formats
Open table formatsNativeDelta Lake native; Apache Iceberg supported through managed Iceberg tables and UniForm. Verify the current state.
Storage/compute separationNative
Query external data in placeNativeLakehouse Federation queries external databases; external locations reach cloud storage.
Compute & Workloads
SQL warehousingNative
Serverless computeNativeServerless SQL warehouses, notebooks, jobs and pipelines.
Spark & notebooksNativeDatabricks was founded by the original creators of Apache Spark.
LanguagesNativeSQL, Python, Scala, R.
Integration & Pipelines
Managed ingestion connectorsNativeLakeflow Connect managed connectors; Auto Loader for files.
Declarative transformation pipelinesNativeLakeflow Declarative Pipelines (formerly Delta Live Tables).
OrchestrationNativeLakeflow Jobs (formerly Databricks Workflows).
Streaming / real-timeNativeSpark Structured Streaming; streaming tables in declarative pipelines.
Governance & Security
Unified catalogNative
LineageNativeColumn-level lineage captured automatically in Unity Catalog.
Row/column-level securityNativeRow filters and column masks in Unity Catalog.
Sharing & Collaboration
Cross-organisation data sharingNativeDelta Sharing, an open protocol that also shares to non-Databricks consumers.
Data marketplaceNative
Data clean roomsNative
Analytics, ML & AI
Native BI / dashboardsNativeAI/BI Dashboards and Genie.
ML lifecycleNativeManaged MLflow, Feature Store, Model Serving.
LLM functions in SQLNativeAI Functions (e.g. ai_query) callable from SQL.
Natural-language querying / data agentsNativeAI/BI Genie spaces; Agent Bricks for building agents.
Operational (OLTP) databasePartialLakebase (managed Postgres). Verify GA status.