Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
Awesome Data Engineering
A curated list of data engineering tools for software developers
igorbarinov/awesome-data-engineering
#awesome
#awesome-list
- List stars
- 9,173
- README repos
- 148
- Indexed repos
- 142
- List commits
- 438
- Forks
- 1,651
- Open issues
- 49
Tracked list growth
GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.
Latest scan 2026-10-17 10:49
Likes history
GitHub starsCommits history
Default branch commits
Latest repo push 2026-10-17
apache/superset
Apache Superset is a Data Visualization and Data Exploration Platform
AI dev
GitHub topics
- Updated
- 2026-10-17
- Lists
- 7 list mentions
- First commit
- 2015-07-02
- License
- Apache-2.0
- Issues
- 586 open
75,176
stars
- Forks
- 18,481
- Commits
- 23,792 commits
- Star growth, last 7 days
- +74 +0.1%
- Commit velocity, last 7 days
- +153 +0.6%
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
AI dev
GitHub topics
- Updated
- 2026-10-17
- Lists
- 7 list mentions
- First commit
- 2014-10-06
- License
- Apache-2.0
- Issues
- 1,932 open
47,194
stars
- Forks
- 18,051
- Commits
- 42,128 commits
- Star growth, last 7 days
- +49 +0.1%
- Commit velocity, last 7 days
- +168 +0.4%
kestra-io/kestra
Event Driven Orchestration & Scheduling Platform for Mission Critical Applications
AI dev
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2019-08-13
- License
- Apache-2.0
- Issues
- 888 open
29,650
stars
- Forks
- 3,405
- Commits
- 13,021 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
AI dev
- Updated
- 2026-10-12
- Lists
- 5 list mentions
- First commit
- 2018-04-30
- License
- Apache-2.0
- Issues
- 2,590 open
16,282
stars
- Forks
- 2,354
- Commits
- 28,117 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
cocoindex-io/cocoindex
Incremental engine for long horizon agents 🌟 Star if you like it!
AI dev
- Updated
- 2026-10-16
- Lists
- 3 list mentions
- First commit
- 2025-03-03
- License
- Apache-2.0
- Issues
- 103 open
11,710
stars
- Forks
- 919
- Commits
- 2,168 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
treeverse/lakeFS
lakeFS - Data version control for your data lake | Git for data
- Updated
- 2026-10-07
- Lists
- 2 list mentions
- First commit
- 2019-09-12
- License
- NOASSERTION
- Issues
- 450 open
5,557
stars
- Forks
- 485
- Commits
- 6,569 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rudderlabs/rudder-server
Privacy and Security focused Segment-alternative, in Golang and React
AI dev
Stack
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2019-07-19
- License
- NOASSERTION
- Issues
- 50 open
4,500
stars
- Forks
- 80
- Commits
- 6,733 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
aws/aws-sdk-pandas
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
- Updated
- 2026-10-15
- Lists
- 2 list mentions
- First commit
- 2019-02-26
- License
- Apache-2.0
- Issues
- 46 open
4,122
stars
- Forks
- 751
- Commits
- 2,842 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
apache/hamilton
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
GitHub topics
- Updated
- 2026-10-14
- Lists
- 3 list mentions
- First commit
- 2019-07-25
- License
- Apache-2.0
- Issues
- 178 open
2,610
stars
- Forks
- 221
- Commits
- 2,116 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
Multiwoven/multiwoven
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- Updated
- 2026-10-15
- Lists
- 3 list mentions
- First commit
- 2023-10-20
- License
- AGPL-3.0
- Issues
- 199 open
1,680
stars
- Forks
- 96
- Commits
- 1,888 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
slothflowlabs/duckle
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- Updated
- 2026-10-14
- Lists
- 3 list mentions
- First commit
- 2026-05-21
- License
- Apache-2.0
- Issues
- 73 open
1,508
stars
- Forks
- 120
- Commits
- 1,751 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
Desbordante/desbordante-core
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
- Updated
- 2026-10-14
- Lists
- 2 list mentions
- First commit
- 2019-07-08
- License
- AGPL-3.0
- Issues
- 90 open
- Forks
- 108
- Commits
- 2,169 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
sealandseacat/dbmask
Discover, mask, and verify sensitive data in SQL databases — an auditable scan → mask → validate workflow for safe database copies.
GitHub topics
- Updated
- 2026-10-11
- Lists
- 2 list mentions
- First commit
- 2026-06-09
- License
- MIT
- Issues
- 3 open
149
stars
- Forks
- 22
- Commits
- 68 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
scriptella/scriptella-etl
Scriptella is an open source ETL (Extract-Transform-Load) and script execution tool written in Java.
GitHub topics
- Updated
- 2026-10-15
- Lists
- 1 list mention
- First commit
- 2006-04-17
- License
- Apache-2.0
- Issues
- 18 open
- Forks
- 50
- Commits
- 895 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
nnnkkk7/snowflake-emulator
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
Stack
- Updated
- 2026-08-17
- Lists
- 2 list mentions
- First commit
- 2025-12-29
- License
- MIT
- Issues
- 10 open
- Forks
- 14
- Commits
- 43 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
drt-hub/drt
Reverse ETL for the code-first data stack
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 1 list mention
- First commit
- 2026-03-27
- License
- Apache-2.0
- Issues
- 53 open
36
stars
- Forks
- 39
- Commits
- 832 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
faucet-hq/faucet-stream
The fast, config-driven way to move data in Rust — a native CLI and an embeddable Rust ETL library
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 2 list mentions
- First commit
- 2026-03-23
- License
- NOASSERTION
- Issues
- 39 open
- Forks
- 8
- Commits
- 509 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
provero-org/provero
Declarative data quality engine. Define checks in YAML, run anywhere.
AI dev
- Updated
- 2026-10-12
- Lists
- 1 list mention
- First commit
- 2026-03-12
- License
- Apache-2.0
- Issues
- 11 open
17
stars
- Forks
- 3
- Commits
- 133 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rbmuller/scherlok
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
GitHub topics
- Updated
- 2026-09-28
- Lists
- 3 list mentions
- First commit
- 2026-04-01
- License
- MIT
- Issues
- 13 open
11
stars
- Forks
- 6
- Commits
- 118 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
godalida/koala-diff
Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.
GitHub topics
- Updated
- 2026-02-25
- Lists
- 1 list mention
- First commit
- 2026-02-13
- License
- MIT
- Issues
- 0 open
- Forks
- 0
- Commits
- 33 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
mgillr/crdt-merge
Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.
- Updated
- 2026-10-04
- Lists
- 1 list mention
- First commit
- 2025-10-01
- License
- NOASSERTION
- Issues
- 7 open
- Forks
- 0
- Commits
- 461 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
aegis-dq/aegis-dq
Open, audit-grade agentic data quality framework with portable industry packs
GitHub topics
- Updated
- 2026-05-27
- Lists
- 2 list mentions
- First commit
- 2026-05-11
- License
- NOASSERTION
- Issues
- 10 open
- Forks
- 2
- Commits
- 76 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dataspoclab/dataspoc-pipe
Data ingestion engine — Singer taps to Parquet in cloud buckets
AI dev
GitHub topics
- Updated
- 2026-04-28
- Lists
- 1 list mention
- First commit
- 2026-03-24
- License
- Apache-2.0
- Issues
- 0 open
- Forks
- 0
- Commits
- 5 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%