Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
Awesome Data Engineering
A curated list of data engineering tools for software developers
igorbarinov/awesome-data-engineering
#awesome
#awesome-list
- List stars
- 9,173
- README repos
- 148
- Indexed repos
- 142
- List commits
- 438
- Forks
- 1,651
- Open issues
- 49
Tracked list growth
GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.
Latest scan 2026-10-17 10:49
Likes history
GitHub starsCommits history
Default branch commits
Latest repo push 2026-10-17
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
GitHub topics
- Updated
- 2026-10-16
- Lists
- 5 list mentions
- First commit
- 2023-07-21
- License
- NOASSERTION
- Issues
- 37 open
62,113
stars
- Forks
- 1,683
- Commits
- 2,312 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
AI dev
GitHub topics
- Updated
- 2026-10-17
- Lists
- 7 list mentions
- First commit
- 2014-10-06
- License
- Apache-2.0
- Issues
- 1,932 open
47,194
stars
- Forks
- 18,051
- Commits
- 42,128 commits
- Star growth, last 7 days
- +49 +0.1%
- Commit velocity, last 7 days
- +168 +0.4%
kestra-io/kestra
Event Driven Orchestration & Scheduling Platform for Mission Critical Applications
AI dev
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2019-08-13
- License
- Apache-2.0
- Issues
- 888 open
29,650
stars
- Forks
- 3,405
- Commits
- 13,021 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
AI dev
- Updated
- 2026-10-12
- Lists
- 5 list mentions
- First commit
- 2018-04-30
- License
- Apache-2.0
- Issues
- 2,590 open
16,282
stars
- Forks
- 2,354
- Commits
- 28,117 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
cocoindex-io/cocoindex
Incremental engine for long horizon agents 🌟 Star if you like it!
AI dev
- Updated
- 2026-10-16
- Lists
- 3 list mentions
- First commit
- 2025-03-03
- License
- Apache-2.0
- Issues
- 103 open
11,710
stars
- Forks
- 919
- Commits
- 2,168 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rudderlabs/rudder-server
Privacy and Security focused Segment-alternative, in Golang and React
AI dev
Stack
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2019-07-19
- License
- NOASSERTION
- Issues
- 50 open
4,500
stars
- Forks
- 80
- Commits
- 6,733 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
aws/aws-sdk-pandas
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
- Updated
- 2026-10-15
- Lists
- 2 list mentions
- First commit
- 2019-02-26
- License
- Apache-2.0
- Issues
- 46 open
4,122
stars
- Forks
- 751
- Commits
- 2,842 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
apache/hamilton
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
GitHub topics
- Updated
- 2026-10-14
- Lists
- 3 list mentions
- First commit
- 2019-07-25
- License
- Apache-2.0
- Issues
- 178 open
2,610
stars
- Forks
- 221
- Commits
- 2,116 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
Multiwoven/multiwoven
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- Updated
- 2026-10-15
- Lists
- 3 list mentions
- First commit
- 2023-10-20
- License
- AGPL-3.0
- Issues
- 199 open
1,680
stars
- Forks
- 96
- Commits
- 1,888 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
slothflowlabs/duckle
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- Updated
- 2026-10-14
- Lists
- 3 list mentions
- First commit
- 2026-05-21
- License
- Apache-2.0
- Issues
- 73 open
1,508
stars
- Forks
- 120
- Commits
- 1,751 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
reductstore/reductstore
High-performance, time-indexed object storage for robotics and industrial IoT
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 1 list mention
- First commit
- 2021-12-23
- License
- Apache-2.0
- Issues
- 15 open
373
stars
- Forks
- 48
- Commits
- 1,639 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
sealandseacat/dbmask
Discover, mask, and verify sensitive data in SQL databases — an auditable scan → mask → validate workflow for safe database copies.
GitHub topics
- Updated
- 2026-10-11
- Lists
- 2 list mentions
- First commit
- 2026-06-09
- License
- MIT
- Issues
- 3 open
149
stars
- Forks
- 22
- Commits
- 68 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
scriptella/scriptella-etl
Scriptella is an open source ETL (Extract-Transform-Load) and script execution tool written in Java.
GitHub topics
- Updated
- 2026-10-15
- Lists
- 1 list mention
- First commit
- 2006-04-17
- License
- Apache-2.0
- Issues
- 18 open
- Forks
- 50
- Commits
- 895 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
drt-hub/drt
Reverse ETL for the code-first data stack
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 1 list mention
- First commit
- 2026-03-27
- License
- Apache-2.0
- Issues
- 53 open
36
stars
- Forks
- 39
- Commits
- 832 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
fulldecent/google-sheets-etl
Live import all your Google Sheets to your data warehouse
GitHub topics
- Updated
- 2026-10-14
- Lists
- 1 list mention
- First commit
- 2019-10-03
- License
- MIT
- Issues
- 3 open
- Forks
- 1
- Commits
- 95 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
faucet-hq/faucet-stream
The fast, config-driven way to move data in Rust — a native CLI and an embeddable Rust ETL library
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 2 list mentions
- First commit
- 2026-03-23
- License
- NOASSERTION
- Issues
- 39 open
- Forks
- 8
- Commits
- 509 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
pujansrt/data-genie
High performant ETL engine written in TypeScript
Stack
GitHub topics
- Updated
- 2026-05-05
- Lists
- 1 list mention
- First commit
- 2025-07-01
- License
- MIT
- Issues
- 1 open
16
stars
- Forks
- 1
- Commits
- 75 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rbmuller/scherlok
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
GitHub topics
- Updated
- 2026-09-28
- Lists
- 3 list mentions
- First commit
- 2026-04-01
- License
- MIT
- Issues
- 13 open
11
stars
- Forks
- 6
- Commits
- 118 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dotflow-io/dotflow
🎲 Dotflow turns an idea into flow! — Lightweight Python library for execution pipelines
GitHub topics
- Updated
- 2026-09-14
- Lists
- 1 list mention
- First commit
- 2024-12-26
- License
- Apache-2.0
- Issues
- 38 open
10
stars
- Forks
- 8
- Commits
- 1,254 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dataspoclab/dataspoc-pipe
Data ingestion engine — Singer taps to Parquet in cloud buckets
AI dev
GitHub topics
- Updated
- 2026-04-28
- Lists
- 1 list mention
- First commit
- 2026-03-24
- License
- Apache-2.0
- Issues
- 0 open
- Forks
- 0
- Commits
- 5 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%