SQLMesh

Overview

SQLMesh is a transformation framework built around one idea: knowing exactly what a change affects, so you rebuild only what genuinely needs rebuilding.

Models are a configuration block naming the kind and schedule, followed by a select. You do not declare dependencies; SQLMesh parses the SQL and works out the lineage itself.

The workflow is deliberately Terraform-like. A plan shows the difference between your code and a target environment, listing added and removed models, direct and indirect changes with diffs, and the date ranges that would be backfilled, before anything touches the warehouse.

Virtual data environments are the headline. Physical tables are keyed by a fingerprint of the model and its upstream dependencies, and a thin layer of views points at them, so promoting to production is a pointer swap rather than a data movement.

Because fingerprints ignore formatting and structural refactoring, cosmetic changes cause no recomputation at all, and work already computed in development can be reused directly in production.

Change categorisation is the other half. A breaking change backfills the model and everything downstream, a non- breaking change backfills only the model itself, and a forward-only change backfills nothing.

Conflicts resolve conservatively: if any upstream change is breaking, the dependent model is treated as breaking. Column-level lineage refines this, so removing a column nothing references is no longer breaking, and the Cloud product extends it further.

Unit tests, audits and column-level lineage are all in the free open-source project rather than behind a paid tier, which is unusual in this category.

It transpiles across more than ten SQL dialects through SQLGlot, its own parser and optimiser, and is explicitly backwards compatible with dbt projects.

One thing to know about the market position: Tobiko Data is now part of Fivetran, which also merged with dbt Labs, so SQLMesh and dbt share a parent company.

Key features and capabilities

The same headings are used for every transformation & modelling entry, so two tools can be read side by side.

How transformations are written

• A model configuration block followed by a select, in SQL, with Python models also supported
• Dependencies are inferred by parsing the SQL, so there is no ref equivalent to maintain
• Macros use an at-prefixed syntax with predefined variables, and Jinja is available too
• Audits attach directly to models and run after evaluation, with statements before or after the main query
• Conventions favour explicit casting and unique, inferrable column names

Execution and materialisation

• Model kinds cover incremental by time range, by unique key, by partition, unmanaged, full, view and embedded
• Also seed models from static files, type 2 slowly changing dimensions, external and managed models
• Incremental models process only missing intervals, which plans surface explicitly before running
• State awareness is the core design rather than an add-on, driven by snapshot fingerprints
• Transformation runs on the warehouse; SQLGlot handles parsing, transpilation and semantic diffing

Testing and data quality

• Unit tests in YAML, asserting on the main query or on individual common table expressions
• Inputs and outputs as YAML, CSV or SQL queries, with partial matching for wide tables
• Tests run automatically whenever a plan is created, as well as on demand
• A generator creates tests from data already in the warehouse
• Audits are configurable data-quality checks run on a different cadence from tests

Environments and change management

• Virtual data environments are shallow clones, computing only the gaps a change actually causes
• Development environments get a schema suffix while production keeps the original names
• Promotion is an atomic view-pointer swap with no data movement, supporting blue-green deployment
• Work computed in development is reused in production, so results match exactly
• Continuous integration through GitHub Actions; Tobiko Cloud adds automatic rollbacks

Documentation and lineage

• Column-level lineage in the open-source project, showing the downstream impact of a schema change
• Impact analysis gives immediate visibility into what a change affects
• Model and column comments carry through as documentation
• A language server and editor extension
• Tobiko Cloud adds a managed catalogue, performance history and per-model warehouse cost tracking

Platform coverage

• Warehouses including BigQuery, Snowflake, Redshift, Databricks and Microsoft Fabric
• Databases including Postgres, MySQL, SQL Server, ClickHouse and Azure SQL
• Engines and local options including Athena, Trino, Spark, DuckDB and MotherDuck
• Tool integrations for dbt, dlt, GitHub Actions and Kestra, with Airflow covered separately

Where it runs and what it costs

• A free command-line tool, with notebook magics and an editor extension
• Tobiko Cloud adds managed scheduling, managed state, isolated Python environments and single sign-on
• Tobiko holds SOC 2
• Self-hosting the commercial product, and Cloud regions, are not published
• Priced as a platform fee plus consumption, explicitly not per seat or project

Pricing

Open sourceFree and Apache 2.0

The open-source framework is Apache 2.0 and free, and unusually it is not feature-crippled: virtual environments, plan and apply, change categorisation, unit tests, audits and column-level lineage are all included. Tobiko Cloud is quote-only, priced as a platform fee plus pay-as-you-go consumption with no seat or project limits, and no figures are published in any currency. There is no public pricing page and no trial length is published, so budgeting means a conversation with sales.

Vendor pricing page →

Demos and videos

• SQLMesh quickstart guide Tutorial · YouTube
• Introducing SQLMesh Product demo · YouTube

About Tobiko Data

Tobiko Data was founded by Iaroslav Zeigerman and brothers Tyson and Toby Mao, with a team drawn from Google, Apple, Airbnb and Netflix; neither its founding year nor its headquarters is published on its own pages. It raised $21.8m in total, a $4.5m seed led by Unusual Ventures and a $17.3m Series A announced in March 2025 led by Theory Ventures, with angels including the chief executives of Fivetran, Census and MotherDuck. It is now part of Fivetran, stated on its own site, though the date and terms are not published there. SQLMesh is Apache 2.0; Tobiko Cloud is the commercial managed product.

tobikodata.com