Data tooling landscape

Managed technical metadata catalogue for data on AWS, used by services such as Athena, EMR and Redshift.

• Cloud service

Managed service for creating and controlling the encryption keys used across AWS services.

• Cloud service

Serverless AWS service for orchestrating multi-step workflows across AWS services.

• Cloud service

Azure's managed service for data integration and ETL/ELT pipelines.

• Cloud service

Data lake storage on Azure Blob Storage, with a hierarchical namespace.

• Cloud service

Microsoft's suite of repos, boards and CI/CD pipelines.

• Commercial

Managed event ingestion service on Azure, with a Kafka-compatible endpoint.

• Cloud service

Azure service for storing secrets, keys and certificates, including customer-managed keys.

• Cloud service

Azure service covering the end-to-end machine learning lifecycle.

• Cloud service

Azure Monitor

Observability Plane · Pipeline & Infrastructure Monitoring

Azure service for collecting and analysing telemetry from cloud and on-premises environments.

• Cloud service

Microsoft's domain-specific language for declaratively deploying Azure resources.

• Open source

Bigeye

Observability Plane · Data Observability

Data observability platform with automated monitoring and lineage-based root cause analysis.

• Commercial

BigID

Data Governance Plane · Privacy & Compliance

Data discovery and classification for privacy, security and governance.

• Commercial

Boomi

Integration & ETL Plane · ELT & Ingestion

Enterprise integration platform spanning application and data integration, API management and agentic workflow orchestration, with an embedded option for building integration into your own product.

• Commercial

ClickHouse

Data Plane · Cloud Data Warehouses

Open-source columnar database for real-time analytics, also offered as a managed cloud service.

• Open core

Hybrid data and AI platform running the same containerised data services on-premises and in cloud, with its own S3-compatible object store built on Apache Ozone.

• Commercial

Coalesce

Integration & ETL Plane · Transformation & Modelling

Visual, code-first data transformation platform for cloud data warehouses.

• Commercial

Data intelligence platform for governance, cataloguing, stewardship and lineage.

• Commercial

Kafka operations platform sitting in front of clusters you already run, pairing a console for visibility and self-service with a proxy that enforces encryption, masking and policy on the wire.

• Commercial

Data streaming platform built around Apache Kafka, available as a cloud service or self-managed.

• Commercial

Cube

Metadata Plane · Semantic Layer & Metrics

Semantic layer that defines metrics once and serves them to BI tools, applications and AI.

• Open core

Cyera

Security Plane · Data Security Posture Management

Data security posture management that discovers and classifies sensitive data across clouds.

• Commercial

Dagster

Control Plane · Orchestration & Scheduling

Data orchestrator built around software-defined assets, with lineage and observability built in.

• Open core

Data Contract CLI

Data Quality Plane · Data Contracts

Command-line tool for linting, testing and exporting data contracts.

• Open source