All Data Engineering Tools

Prefect is a workflow orchestration framework for building resilient data...

An orchestration platform for the development, production, and observation...

data load tool (dlt) is an open source Python library that makes data...

Open Source Feature Flags, Experimentation, and Product Analytics

Python ETL framework for stream processing, real-time analytics, LLM...

Python Client for Supabase. Query Postgres from Flask, Django, FastAPI....

Convert documents to structured data effortlessly. Unstructured is...

ingestr is a CLI tool to copy data between any databases with a single...

Transform, query, and download geospatial data on the web.

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Meltano: the declarative code-first data integration engine that powers your...

Clean APIs for data cleaning. Python implementation of R package Janitor

Quilt is a Scientific Data Management Platform on AWS that helps teams and...

A dbt adapter for Databricks.

Change data capture for a variety of databases. Please log issues at...

Flink CDC is a streaming data integration tool

The leading data integration platform for ETL / ELT data pipelines from...

Apache Superset is a Data Visualization and Data Exploration Platform

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability,...

SeaTunnel is a multimodal, high-performance, distributed, massive data...

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML,...

Relational data pipelines for the science lab

OpenMetadata is a unified metadata platform for data discovery, data...

Apache DevLake is an open-source dev data platform to ingest, analyze, and...

CrateDB is a distributed and scalable SQL database for storing and analyzing...

Blazing-fast Data-Wrangling toolkit

locopy: Loading/Unloading to Redshift and Snowflake using Python.

A high-performance observability data pipeline.

lakeFS - Data version control for your data lake | Git for data

A unified interface for distributed computing. Fugue executes SQL, Python,...

Free universal database tool and SQL client

A local-first workflow engine built the way it should be: declarative,...

Data pipelines for cloud config and security data. Build cloud asset...

Event streaming platform for agents, apps, and analytics. Continuously...

Fast, Simple and a cost effective tool to replicate data from Postgres to...

Hop Orchestration Platform

phpMyFAQ - Open Source FAQ web application for PHP 8.3+ and MySQL,...

The Public Utility Data Liberation Project provides analysis-ready energy...

A utility library for comparing and synchronizing different datasets.

The leader in Customer Data Infrastructure

The AI-Native Infrastructure for Enterprise Apps. Powered by ObjectStack...

Wikidata and Wiktionary language data extraction

An extensible framework for Personal Data Management.

Command-line interface for downloading WARN Act notices of qualified plant...

Scalable and efficient data transformation framework - backwards compatible with dbt.

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL...

Dataform is a framework for managing SQL based data operations in BigQuery

Egeria core

Open-source data framework for biology. Context and memory for datasets and...

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream,...

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg or Plain...

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

Supplementary Materials for the The Complete dbt (Data Build Tool) Bootcamp...

A fast and versatile ETL tool that can transfer data between RDBMS and NoSQL...

Database Reporting Tool and Tasks (.Net)

Know your data better！Datavines is Next-gen Data Observability Platform,...

A Spark plugin for reading and writing Excel files

KNIME Analytics Platform

Automation fetching data for Vietlott. Just for fun.

Inspection of tabular (csv, xls-like) files to guess the columns' content

Data Contracts engine for the modern data stack. https://www.soda.io

A compute manifest and composable tools for data, built on Ibis, DataFusion,...

Apache Hamilton helps data scientists and engineers define testable,...

Fancy stream processing made operationally mundane

The dbt-native data observability solution for data & analytics engineers....

log data pre processing, generation and shipping in python

A system for agentic LLM-powered data processing and ETL

Save or export your private and public Slack messages, threads, files, and...

A wrapper around the python-irodsclient to allow for easy interaction with...

ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data...

Java dataframe and visualization library

Use data to tell stories.An intelligent Visualization Narrative Development...

The premier open source Data Quality solution

dbt-sugar is a CLI tool that allows users of dbt to have fun and ease...

A Python package to load complex XML files into a relational database

A query / expression language for performing computations on JSON-like...

Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No...

Data Engineering Zoomcamp is a free 9-week course on building...

HeavyDB (formerly MapD/OmniSciDB)

Sling is a CLI tool that extracts data from a source storage/database and...

Privacy and Security focused Segment-alternative, in Golang and React

Consolidate, enrich and republish the data gathered by warn-scraper

Easy-to-use and high-performance TypeScript library for data analysis. Works...

1 Line of code data quality profiling & exploratory data analysis for Pandas...

⚡ Fastest SQL ETL pipeline in a single C++ binary, built for stream...

A lightweight, developer-focused database management tool. Supports MySQL,...

Apache Wayang is the first cross-platform data processing system.

Server application to serve U.S. federal spending data via a RESTful API

visual data prep powered by python

Snowflake Snowpark Python API

🎲 Business Logic Code in a flow!

Desbordante is a high-performance data profiler that is capable of...

ReplicaDB is open source tool for database replication, designed for...

Use SQL to instantly query AWS resources across regions and accounts. Open...

A collection of tools for extracting FHIR resources and analytics services...

LangChain abstractions backed by Postgres Backend

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

The best place to learn data engineering. Built and maintained by the data...

First open-source data discovery and observability platform. We make a life...