Kestra

Overview

Kestra takes the opposite authoring stance to Airflow, Dagster and Prefect: workflows are declarative YAML, not Python. Task code can be written in Python, shell or other languages through plugins, so the orchestrator is language-agnostic and not limited to Python teams.

It is event-driven as much as scheduled. Alongside cron schedules with backfill, it offers flow triggers that fire when upstream flows finish, webhook and polling triggers, and realtime triggers that react within milliseconds to a message from Kafka or SQS.

Architecturally it runs on the JVM and splits a stateless control plane from a data plane of workers, which is what allows hybrid deployments where execution stays in your network.

It leans further into AI than its peers: an MCP server so agents can drive it, a trigger that exposes a flow as a callable tool for an agent, and a copilot that writes and edits flows from plain language.

Two caveats matter when comparing. First, its asset and lineage features, which give it data awareness and OpenLineage export, are Enterprise and Cloud only, not in the open-source edition. Second, quite a lot else is paid too: worker groups, unit tests, audit logs, multi-tenancy, single sign-on and role-based access.

The open-source core is Apache 2.0 and genuinely usable in production on PostgreSQL or MySQL. Kestra claims more than 2,000 plugins, covering Spark, Kafka, Snowflake, BigQuery, the major clouds and AI services.

Key features and capabilities

The same headings are used for every orchestration & scheduling entry, so two tools can be read side by side.

How pipelines are written

• Declarative YAML flows, with tasks carrying retries, timeouts and conditions
• Flowable tasks express orchestration logic; runnable tasks do the work on a worker
• Task code in Python, shell and other languages through plugins
• Code and no-code editors, a playground for iterating, and VS Code support

Scheduling and triggers

• Cron schedules with calendar conditions and backfill
• Flow triggers that fire when upstream flows finish in a matching state
• Webhook and polling triggers
• Realtime triggers with millisecond latency from Kafka or SQS, and an MCP tool trigger

Execution and scaling

• Stateless control plane and a data plane of workers, enabling hybrid deployments
• Standalone single process, or distributed components scaled independently
• Task runners: process, Docker and Kubernetes in open source; ECS, Azure Batch and Google Batch documented
• Worker groups and worker isolation are Enterprise features

Monitoring and recovery

• Interface with flows, executions, logs and dashboards
• Gantt and topology views per execution
• Custom dashboards and a blueprint library
• Audit logs, log shipping and cluster monitoring are Enterprise

Data awareness

• Assets name the resources a flow reads and writes, building a lineage graph
• Freshness states show whether an asset is fresh, stale or failed
• OpenLineage export to DataHub, Marquez or Atlan
• Assets and lineage are Enterprise and Cloud only, not open source

Integrations

• More than 2,000 plugins claimed
• Spark, Kafka, Flink, Airbyte, Snowflake, BigQuery and JDBC targets
• AWS, Azure and Google Cloud services; Kubernetes, Docker, Ansible and Argo CD
• Versioned plugins and allow-lists are Enterprise features

How it runs

• Self-hosted on the JVM, standalone or distributed
• PostgreSQL or MySQL by default; Kafka and Elasticsearch backends are Enterprise
• Kestra Cloud managed in US and EU regions on Google Cloud
• Apache 2.0 core; Enterprise edition is proprietary

Pricing

Open source

The open-source edition is free under Apache 2.0 and includes unlimited flows and executions, the plugin ecosystem, event-driven scheduling and the MCP server. Kestra Cloud and Enterprise publish no prices at all: Cloud is pay-as-you-scale by request, Enterprise is an annual subscription per instance quoted by sales. No trial length is published. Note that assets, lineage, single sign-on, roles, multi-tenancy and audit logs are all paid-only.

Vendor pricing page →

Demos and videos

• Getting started with Kestra: build your first workflow Tutorial · YouTube
• Kestra guided tour Product demo · YouTube

About Kestra

Kestra was founded in 2021 by Emmanuel Darras and Ludovic Dehon and has raised $36m in total, including a $25m Series A. Its angel investors include founders from Algolia, Datadog, dbt Labs, Airbyte, Hugging Face, Talend and Collibra. It is remote-first and does not publish a head office, with a team of roughly 60. The core is Apache 2.0 and single-vendor rather than foundation-governed, with a proprietary Enterprise edition and a managed cloud.

Founded 2021 · kestra.io