Prefect

Overview

Prefect turns ordinary Python functions into orchestrated workflows. You decorate a function as a flow and its steps as tasks, and that is most of the learning curve: no domain-specific language, no YAML, no special syntax.

Its distinguishing trait is how dynamic it is. Tasks can be created at runtime based on the data actually seen, which suits workflows whose shape is not known in advance, and is part of why it is often chosen for machine learning and agent workloads rather than fixed nightly batches.

Workflows are packaged as deployments for scheduling and remote running. Schedules can be cron, interval or calendar rules with time zones, and flows can also be triggered by events or the API.

Infrastructure is decoupled through work pools and workers. Pull pools use a worker you run inside your network; push pools submit work straight to a serverless provider with no worker at all, covering Kubernetes, ECS, Azure Container Instances, Cloud Run, Vertex AI, Modal and Prefect's own managed compute.

It understands data to a degree: a materialize decorator records what a flow produced, keyed by URI and grouped by scheme, with dependencies inferred from the task graph. It is lighter than Dagster's asset model, and it does not emit OpenLineage.

The framework is Apache 2.0, with Prefect Cloud as the managed control plane. Prefect also announced in July 2026 that it is acquiring Dagster Labs, so both products now sit under one company, with each continuing separately.

Key features and capabilities

The same headings are used for every orchestration & scheduling entry, so two tools can be read side by side.

How pipelines are written

• Plain Python with flow and task decorators; no DSL or YAML
• Tasks can be created dynamically at runtime from real data
• Workflows packaged as deployments for scheduling and remote runs
• A materialize decorator declares what a flow produced

Scheduling and triggers

• Cron, interval and calendar-rule schedules, all time-zone aware
• Several schedules per deployment, each able to be deactivated without deleting it
• Event and API triggers
• Catch-up and backfill are not documented on the schedules page

Execution and scaling

• Work pools and workers separate orchestration from infrastructure
• Pull pools run a worker you host; push pools submit directly to a serverless provider
• Pool types include process, Docker, Kubernetes, ECS, Azure, Cloud Run, Vertex AI and Modal
• State tracking with retries and caching

Monitoring and recovery

• Interface for run history and scheduling, self-hosted or in Prefect Cloud
• Event-based automations drive alerting, with tier-based quotas
• Asset health shown as succeeded, failed or not yet recorded
• Run retention is 7 or 14 days depending on tier

Data awareness

• Assets recorded through a materialize decorator, keyed by URI
• Grouped automatically by scheme and path, giving a workspace-wide view
• Dependencies inferred from the task graph, or declared for external systems
• No OpenLineage emission or built-in data quality checks are published

Integrations

• Around 20 integration libraries, distributed as Python packages
• AWS, Azure, Google Cloud, Databricks, Snowflake, dbt, Docker, Kubernetes, Dask and Ray
• Fivetran, GitHub, GitLab, Slack and SQLAlchemy among the rest
• Spark is not a first-party integration

How it runs

• Self-hosted server, with Docker Compose and Kubernetes Helm charts
• PostgreSQL 14.9 or later for multi-server; SQLite for single-server only
• Prefect Cloud as the managed control plane, with hybrid or managed compute
• Apache 2.0 for the framework

Pricing

SubscriptionFree tier; Starter $100 a month

Hobby is free forever: two users, five deployments and 500 serverless minutes a month. Starter is $100 a month flat for three users, 20 deployments and 75 hours of serverless compute. Team is $100 per user a month for four to eight users, with 100 deployments, 225 hours and a 24-hour audit log. Enterprise is quoted and adds multiple workspaces, custom limits and a 99.99% uptime commitment. The framework itself is free under Apache 2.0.

Vendor pricing page →

Demos and videos

• Prefect basics: write workflows you can schedule and monitor Tutorial · YouTube
• Prefect 3.0: a framework to build resilient pipelines Product demo · YouTube

About Prefect

Prefect Technologies was founded in 2018 by Jeremiah Lowin and is based in Washington, D.C. as a remote-first company. Its published funding includes a Series A led by Positive Sum and a $32m Series B in 2021 from Tiger Global and Bessemer Venture Partners. The framework is Apache 2.0, single-vendor rather than foundation governed, and the company also publishes FastMCP. In July 2026 it announced the acquisition of Dagster Labs, saying nothing changes for users of either product.

Founded 2018 · Washington, D.C. · prefect.io