The Linux Foundation Projects
Delta Lake
logo

Delta Lake Integrations

Use the following frameworks, Delta Sharing clients, managed services, and/or community integrations
for Delta Lake and Delta Sharing.

UniForm

Delta Universal Format (UniForm)
allows you to read Delta tables with Iceberg and Hudi clients

Requirements Enable UniForm

Frameworks

Use the following frameworks and languages including but not limited to Apache Flink, Apache Spark, Trino, and Rust.

Apache Druid

docs source code

Druid

This connector allows Apache Druid to read from Delta Lake.

Apache Flink

docs source code

Flinkstandalone

This connector allows Apache Flink to write to Delta Lake.

Apache Hive

docs source code

Hivestandalone

This connector allows Apache Hive to read from Delta Lake.

Apache Pulsar

docs source code

Pulsarcommunity

This connector allows Apache Pulsar to read from and write to Delta Lake.

Apache Spark™

docs source code

Spark

This connector allows Apache Spark™ to read from and write to Delta Lake.

ClickHouse

docs source code

ClickHouse

ClickHouse is a column-oriented database that allows users to run SQL queries on Delta Lake tables. This provides a read-only integration with existing Delta Lake tables in Amazon S3.

Dagster

docs source code

DagsterPython

Use the Delta Lake IO Manager to read from and write to Delta Lake in your Dagster orchestration pipelines.

Delta Rust API

docs source code

RustPython

This library allows Rust (with Python bindings) low level access to Delta tables and is intended to be used with data processing frameworks like datafusion, ballista, rust-dataframe, vega, etc.

Delta Standalone

docs source code

ScalaJavastandalone

This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.

FINOS Legend

docs source code

FINOSLegend

An extension to the FINOS Legend framework for Apache Spark™ / Delta Lake based environment, combining best of open data standards with open source technologiesThis connector allows Trino to read from and write to Delta Lake.

Hopsworks

docs source code

HopsworksPython

Hopsworks Feature Store stores, manages, and serves feature data in Delta Lake.

Kafka Delta Ingest

docs source code

KafkaRust

This project builds a highly efficient daemon for streaming data through Apache Kafka into Delta Lake.

PrestoDB

docs source code

PrestoDBstandalone

This connector allows PrestoDB to read from Delta Lake.

RisingWave

docs source code

RisingWavestandalone

This connector allows RisingWave to write to Delta Lake.

SQL Delta Import

docs source code

SQLJDBC

This utility is for importing data from a JDBC source into a Delta Lake table.

StarRocks

docs source code

StarRocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. StarRocks has the ability to read from Delta Lake.

Trino

docs source code

Trino

This connector allows Trino to read from and write to Delta Lake.

Sharing

Use the following clients that integrate with Delta Sharing from C++ to Rust.

C++

docs source code

C++Delta Sharingcommunity

This connector allows a C++ client to read from Delta Sharing endpoint.

Excel

docs

ExcelDelta Sharing

This connector allows a Excel client to read from Delta Sharing endpoint.

Go

source code

GoDelta Sharingcommunity

This connector allows a Go client to read from Delta Sharing endpoint.

Java

source code

JavaDelta Sharingcommunity

This connector allows a Java client to read from Delta Sharing endpoint.

Kotosiro Sharing

source code

KotosiroDelta SharingRust

A Minimalistic Rust Implementation of Delta Sharing Server.

MLflow

docs source code

MLflowDelta Sharingcommunity

Perform model exchange via Delta Sharing and MLflow

node.js

docs source code

node.jsDelta Sharingcommunity

This connector allows node.js to read from Delta Sharing endpoint.

Oracle

OracleDelta Sharing

This connector allows for Delta Sharing with Oracle Autonomous Database Data Studio.

Power BI

docs

PowerBIDelta Sharing

This connector allows Power BI to read from Delta Sharing endpoint.

R

source code

RDelta Sharingcommunity

This connector allows a R client to read from Delta Sharing endpoint.

Rust

docs source code

RustDelta Sharingcommunity

This connector allows a Rust client to read from Delta Sharing endpoint.

Terminal

source code

TerminalDelta Sharingcommunity

Terminal application for browsing Delta Sharing Metadata.

Services

Use the managed services of your choice that integrate with Delta Lake.

Athena

docs

AthenaAWS

This utility allows Athena to natively read from Delta Lake starting with Athena SQL 3.0

AWS EMR

docs

AWSEMR

Starting with Amazon EMR release 6.9.0, you can use Apache Spark 3.x on Amazon EMR clusters with Delta Lake tables.

AWS Glue

docs

AWSGlue

AWS Glue 3.0 and later supports the Linux Foundation Delta Lake framework

aws-pandas-sdk

docs source code

pandasAWSawswrangleraws-pandas-sdk

pandas on AWS - Easy integration with AWS services including optional dependency with Delta Lake

Azure Stream Analytics

docs

AzureASAStream Analytics

Azure Stream Analytics provides native write support for Delta Lake

BigQuery

docs

BigQuery’s native Delta Lake support enables seamless delivery of data for downstream applications.

Databricks

docs

DatabricksAzureGCPAWS

Delta Lake is included within Databricks allowing it to read from and write to Delta Lake.

Microsoft Fabric

docs

In order to achieve seamless data access across all compute engines in Microsoft Fabric, Delta Lake is chosen as the unified table format.

Power BI

docs source code

PowerBIcommunity

This connector allows Power BI to read from Delta Lake.

Redshift

docs source code

RedshiftAWSmanifest

This utility allows AWS Redshift to read from Delta Lake using a manifest file.

Snowflake (Beta)

docs

Snowflake

This preview allows Snowflake to read from Delta Lake via an external table.

Starburst

docs

StarburstAzureGCPAWS

The Starburst Delta Lake connector is an extended version of the Trino/Delta Lake connector with configuration and usage identical.

StarTree

docs

StarTreePinot

StarTree Cloud includes the Apache Pinot / Delta Lake connector.

Community

Try out the following community integrations with Delta Lake.

Apache Beam

docs source code

Beamstandalonecommunity

This connector allows Apache Beam to read from Delta Lake.

Athena Query Federation (Beta)

docs source code

AWSAthenastandalonecommunity

This connector allows AWS Athena to read from Delta Lake.

Beam Delta Lake

docs source code

Beamstandalonecommunity

With DataLakeIO, data from Apache Beam's pipelines can be read from and written to Delta Lake

Ceph

source code

Cephcommunity

This connector allows you to read and write from Delta tables on Ceph storage.

dlt | SparkR

docs source code

SparkRcommunity

This package allows SparkR to read from and write to Delta Lake.

DataHub

source code

DataHubcommunity

This connector allows DataHub to extract Delta Lake metadata.

Datastream Connector

source code

GCSDatastreambadal.iocommunity

As Datastream streams changes to files to Google Cloud Storage, this connector streams these files and writes the changes to Delta Lake.

MinIO

docs source code

MinIOcommunity

This connector allows you to read and write from Delta tables on MinIO storage.