Apache Spark
Data Plane · Query & Processing Engines
Distributed engine for large-scale data processing, SQL, streaming and machine learning.
- Open source
Data Plane · Query & Processing Engines
In-process analytical SQL database, popular for local analytics and embedded workloads.
DuckDB is an analytical SQL database that runs inside your own process. There is no server to start and no cluster to size: it is embedded completely within the host program, which is why people describe it as SQLite for analytics.
The engine is columnar and vectorised. Queries are interpreted, but a large batch of values, a vector, is processed in one operation, which is what gives it warehouse-like speed on a laptop.
It is a real database, not a query tool. It provides ACID guarantees through a custom multi-version concurrency control design that the project describes as bulk-optimised.
Parallelism is based on row groups of 122,880 rows by default, which matters in practice: a query needs at least that many rows per thread before extra cores help at all.
Larger-than-memory work is supported by spilling to disk, with group-by, join, order-by and window operations all able to run out of core.
Read the published caveats, because they are unusually honest. Several blocking operators in one query can still exhaust memory, list and string aggregation do not spill, and the docs warn against read-write use over network-attached storage.
One correction worth making explicitly: the FAQ states DuckDB is not an in-memory database, despite how it is often described.
Reach comes from extensions. Around 29 core extensions cover Parquet, JSON, Avro, Excel, Delta, Iceberg, DuckLake, Unity Catalog, plus httpfs, AWS and Azure storage and direct Postgres, MySQL and SQLite access.
Licensing is clean and unusually generous: MIT, with the intellectual property held by a non-profit foundation and no enterprise edition at all.
Scaling beyond one machine means MotherDuck, the commercial service built on it, which adds dual execution running query stages locally, in the cloud, or across both.
The same headings are used for every query & processing engines entry, so two tools can be read side by side.
Open sourceFree, MIT licensed
DuckDB is MIT-licensed, free, and the FAQ states there is no enterprise edition, so there is no paid tier to compare. Support services are sold separately by DuckDB Labs. If you need it hosted, MotherDuck publishes real numbers, a Lite plan from zero with 10 GB of storage and 10 hours of compute included, a Business plan at $250 per organisation a month plus usage, compute from $0.60 a unit hour metered per query up to $24.00 an hour on the largest size, and storage at $0.04 a GB a month.
DuckDB was created by Mark Raasveldt and Hannes Mühleisen at the Centrum Wiskunde & Informatica in Amsterdam. The intellectual property and trademarks sit with Stichting DuckDB Foundation, a Dutch non-profit whose board is Mühleisen, Raasveldt and Peter Boncz, deliberately separating ownership from the company. DuckDB Labs is the commercial entity that employs core contributors and sells support. The foundation is funded by membership dues, gold members paying at least €100,000 a year and including AWS, MotherDuck and Posit. Everything is MIT-licensed, and the FAQ states there is no enterprise version.
Founded 2018 · Amsterdam, Netherlands · duckdb.org
Data Plane · Query & Processing Engines
Distributed engine for large-scale data processing, SQL, streaming and machine learning.
Data Plane · Query & Processing Engines
Lakehouse compute engine that allocates compute per query rather than by fixed warehouse size, aimed at high-concurrency SQL and AI workloads with no data movement.

Data Plane · Query & Processing Engines
Fast DataFrame library written in Rust, with Python bindings.
Data Plane · Query & Processing Engines
Distributed SQL query engine for querying data where it lives, across many sources.
Drafted with AI assistance and checked against the vendor’s own documentation.