Data Plane · Query & Processing Engines

DuckDB

In-process analytical SQL database, popular for local analytics and embedded workloads.

Overview

DuckDB is an analytical SQL database that runs inside your own process. There is no server to start and no cluster to size: it is embedded completely within the host program, which is why people describe it as SQLite for analytics.

The engine is columnar and vectorised. Queries are interpreted, but a large batch of values, a vector, is processed in one operation, which is what gives it warehouse-like speed on a laptop.

It is a real database, not a query tool. It provides ACID guarantees through a custom multi-version concurrency control design that the project describes as bulk-optimised.

Parallelism is based on row groups of 122,880 rows by default, which matters in practice: a query needs at least that many rows per thread before extra cores help at all.

Larger-than-memory work is supported by spilling to disk, with group-by, join, order-by and window operations all able to run out of core.

Read the published caveats, because they are unusually honest. Several blocking operators in one query can still exhaust memory, list and string aggregation do not spill, and the docs warn against read-write use over network-attached storage.

One correction worth making explicitly: the FAQ states DuckDB is not an in-memory database, despite how it is often described.

Reach comes from extensions. Around 29 core extensions cover Parquet, JSON, Avro, Excel, Delta, Iceberg, DuckLake, Unity Catalog, plus httpfs, AWS and Azure storage and direct Postgres, MySQL and SQLite access.

Licensing is clean and unusually generous: MIT, with the intellectual property held by a non-profit foundation and no enterprise edition at all.

Scaling beyond one machine means MotherDuck, the commercial service built on it, which adds dual execution running query stages locally, in the cloud, or across both.

Key features and capabilities

The same headings are used for every query & processing engines entry, so two tools can be read side by side.

Execution model
  • Single-node and in-process, embedded in the host program rather than run as a server
  • Columnar vectorised execution, processing a batch of values in one operation
  • ACID transactions through a bulk-optimised multi-version concurrency control design
  • Parallelism across row groups of 122,880 rows, with a configurable thread count
Languages and APIs
  • SQL is the primary interface
  • Primary clients, C, the CLI, Java, Go, Node.js, ODBC, Python, R, Rust and WebAssembly
  • Secondary clients, Arrow ADBC, C# and C++
  • Primary clients get new features first; Node.js, R and C# have external maintainers
Data sources and formats
  • File formats through core extensions, Parquet, JSON, Avro, Excel and encodings
  • Table formats, Delta, Iceberg, DuckLake, Lance and Vortex, plus Unity Catalog
  • Remote and object storage through httpfs, AWS and Azure extensions
  • Direct querying of Postgres, MySQL, SQLite and ODBC sources
Performance and scale
  • Larger-than-memory workloads spill to disk, with a configurable temporary directory
  • Group-by, join, order-by and window operations all run out of core
  • Several blocking operators in one query can still exhaust memory; list and string aggregation do not spill
  • The project publishes its own benchmarks, including TPC-H at scale factor 100 running on mobile phones
What it suits
  • Analytical queries on one machine, from edge devices to servers with hundreds of cores
  • Local and embedded analytics, including in the browser through WebAssembly
  • Not an in-memory database, and read-write use over network-attached storage is advised against
  • Not distributed; scaling out means the commercial MotherDuck service or lakehouse formats
Ecosystem
  • Core and community extensions, including spatial, full-text search, vector search and a local UI
  • Related foundation projects, DuckLake and Quack
  • Corporate backing through foundation gold members AWS, MotherDuck and Posit
  • Compiles for Linux, macOS and Windows on both x86 and ARM
Where it runs and what it costs
  • Installed as a library inside your own process, or used through the CLI binary
  • Runs in the browser through WebAssembly
  • MotherDuck is the managed commercial option, with dual execution across laptop and cloud
  • No server, cluster or Kubernetes deployment to operate

Pricing

Open sourceFree, MIT licensed

DuckDB is MIT-licensed, free, and the FAQ states there is no enterprise edition, so there is no paid tier to compare. Support services are sold separately by DuckDB Labs. If you need it hosted, MotherDuck publishes real numbers, a Lite plan from zero with 10 GB of storage and 10 hours of compute included, a Business plan at $250 per organisation a month plus usage, compute from $0.60 a unit hour metered per query up to $24.00 an hour on the largest size, and storage at $0.04 a GB a month.

Vendor pricing page →

Demos and videos

About DuckDB Foundation and DuckDB Labs

DuckDB was created by Mark Raasveldt and Hannes Mühleisen at the Centrum Wiskunde & Informatica in Amsterdam. The intellectual property and trademarks sit with Stichting DuckDB Foundation, a Dutch non-profit whose board is Mühleisen, Raasveldt and Peter Boncz, deliberately separating ownership from the company. DuckDB Labs is the commercial entity that employs core contributors and sells support. The foundation is funded by membership dues, gold members paying at least €100,000 a year and including AWS, MotherDuck and Posit. Everything is MIT-licensed, and the FAQ states there is no enterprise version.

Founded 2018 · Amsterdam, Netherlands · duckdb.org

Other query & processing engines tools

Apache Spark

Data Plane · Query & Processing Engines

Distributed engine for large-scale data processing, SQL, streaming and machine learning.

  • Open source

e6data

Data Plane · Query & Processing Engines

Lakehouse compute engine that allocates compute per query rather than by fixed warehouse size, aimed at high-concurrency SQL and AI workloads with no data movement.

  • Commercial

Polars

Data Plane · Query & Processing Engines

Fast DataFrame library written in Rust, with Python bindings.

  • Open source

Trino

Data Plane · Query & Processing Engines

Distributed SQL query engine for querying data where it lives, across many sources.

  • Open source

Drafted with AI assistance and checked against the vendor’s own documentation.