Data tooling landscape

A neutral map of the tools used to build data platforms, organised by the plane they serve: from orchestration and storage to governance, security and AI.

51 tools · 10 planes · 36 categories. Listings are editorial and unpaid; any sponsored placement will always be labelled.

Decides what runs, when and how. Orchestration, infrastructure as code and CI/CD for data.

Orchestration & Scheduling

DataOps & CI/CD

Data Plane

17 tools

Where data is stored and processed. Warehouses, lakehouses, open table formats, engines, storage and streaming.

Moves and shapes data. Ingestion connectors, change data capture, transformation and reverse ETL.

Change Data Capture & Replication

Transformation & Modelling

Reverse ETL & Activation

    Proves data is fit for use. Tests, validation rules, data contracts and profiling.

    Testing & Validation

    Profiling & Anomaly Detection

      Tells you when data or pipelines break, and what they cost. Data observability, monitoring and FinOps.

      Data Observability

        Pipeline & Infrastructure Monitoring

        Cost & FinOps

          Describes the data. Catalogues, technical catalogues, lineage and semantic layers.

          Data Catalogues & Discovery

          Table & Technical Catalogues

          Semantic Layer & Metrics

            Decides who may use which data, and for what. Access policies, stewardship, privacy and master data.

            Access Policy Enforcement

              Privacy & Compliance

                Master Data Management

                Protects data and access. Identity, secrets and keys, data security posture, masking and tokenisation.

                Secrets & Key Management

                Data Security Posture Management

                  Masking & Tokenisation

                    Turns data into decisions. BI and dashboards, notebooks and data apps.

                    Notebooks & Data Apps

                      Builds on governed data. ML platforms, feature stores and vector search.

                      ML Platforms & Lifecycle

                        Feature Stores

                          Vector Databases & Search

                            All tools

                            Planes
                            Categories

                            Control Plane

                            Data Plane

                            Integration & ETL Plane

                            Data Quality Plane

                            Observability Plane

                            Metadata Plane

                            Data Governance Plane

                            Security Plane

                            Visualisation & BI Plane

                            AI & ML Plane

                            51 results · showing 1–24

                            Alation

                            Metadata Plane · Data Catalogues & Discovery

                            Enterprise data catalogue for search, discovery, stewardship and governance.

                            • Commercial

                            Amazon S3

                            Data Plane · Object Storage

                            Object storage service from AWS, commonly used as the foundation of data lakes.

                            • Cloud service

                            Apache Airflow

                            Control Plane · Orchestration & Scheduling

                            Open-source platform for authoring, scheduling and monitoring workflows defined as Python DAGs.

                            • Open source

                            Apache Hudi

                            Data Plane · Open Table Formats

                            Open data lakehouse format focused on incremental processing, upserts and deletes.

                            • Open source

                            Apache Iceberg

                            Data Plane · Open Table Formats

                            Open table format for large analytic tables, supported by many engines and platforms.

                            • Open source

                            Apache Kafka

                            Data Plane · Streaming & Stream Processing

                            Distributed event streaming platform for high-throughput data pipelines and streaming applications.

                            • Open source

                            Apache Spark

                            Data Plane · Query & Processing Engines

                            Distributed engine for large-scale data processing, SQL, streaming and machine learning.

                            • Open source

                            Atlan

                            Metadata Plane · Data Catalogues & Discovery

                            Active metadata platform for data cataloguing, discovery and governance.

                            • Commercial

                            AWS Glue

                            Integration & ETL Plane · ELT & Ingestion

                            Serverless data integration service on AWS for discovering, preparing and moving data.

                            • Cloud service

                            Azure Data Factory

                            Integration & ETL Plane · ELT & Ingestion

                            Azure's managed service for data integration and ETL/ELT pipelines.

                            • Cloud service

                            Azure Data Lake Storage

                            Data Plane · Object Storage

                            Data lake storage on Azure Blob Storage, with a hierarchical namespace.

                            • Cloud service

                            Azure DevOps

                            Control Plane · DataOps & CI/CD

                            Microsoft's suite of repos, boards and CI/CD pipelines.

                            • Commercial

                            Azure Event Hubs

                            Data Plane · Streaming & Stream Processing

                            Managed event ingestion service on Azure, with a Kafka-compatible endpoint.

                            • Cloud service

                            Azure Key Vault

                            Security Plane · Secrets & Key Management

                            Azure service for storing secrets, keys and certificates, including customer-managed keys.

                            • Cloud service

                            Azure Monitor

                            Observability Plane · Pipeline & Infrastructure Monitoring

                            Azure service for collecting and analysing telemetry from cloud and on-premises environments.

                            • Cloud service

                            Bicep

                            Control Plane · Infrastructure as Code

                            Microsoft's domain-specific language for declaratively deploying Azure resources.

                            • Open source

                            ClickHouse

                            Data Plane · Cloud Data Warehouses

                            Open-source columnar database for real-time analytics, also offered as a managed cloud service.

                            • Open core

                            Collibra

                            Data Governance Plane · Governance & Stewardship

                            Data intelligence platform for governance, cataloguing, stewardship and lineage.

                            • Commercial

                            Data Factory in Microsoft Fabric

                            Integration & ETL Plane · ELT & Ingestion

                            Pipelines and Dataflow Gen2 for ingesting and transforming data in Microsoft Fabric.

                            • Cloud service

                            Databricks

                            Data Plane · Lakehouse Platforms

                            Lakehouse platform built on Apache Spark and open table formats, governed through Unity Catalog.

                            • Commercial

                            Databricks Asset Bundles

                            Control Plane · Infrastructure as Code

                            Defines Databricks jobs, pipelines and resources as code, deployed with the Databricks CLI.

                            • Commercial

                            dbt

                            Integration & ETL Plane · Transformation & Modelling

                            SQL-based transformation framework with testing and documentation, plus a managed platform.

                            • Open core

                            Debezium

                            Integration & ETL Plane · Change Data Capture & Replication

                            Open-source change data capture platform that streams database changes into Kafka and other systems.

                            • Open source

                            Delta Lake

                            Data Plane · Open Table Formats

                            Open table format that adds ACID transactions, time travel and schema enforcement to data lakes.

                            • Open source

                            Missing a tool?

                            The landscape is curated editorially for now; public submissions open later. Inclusion is never paid for, and any sponsored placement will be clearly labelled.