Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
Awesome Data Engineering
A curated list of data engineering tools for software developers
igorbarinov/awesome-data-engineering
#awesome
#awesome-list
- List stars
- 9,173
- README repos
- 148
- Indexed repos
- 142
- List commits
- 438
- Forks
- 1,651
- Open issues
- 49
Tracked list growth
GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.
Latest scan 2026-10-17 10:49
Likes history
GitHub starsCommits history
Default branch commits
Latest repo push 2026-10-17
treeverse/lakeFS
lakeFS - Data version control for your data lake | Git for data
- Updated
- 2026-10-07
- Lists
- 2 list mentions
- First commit
- 2019-09-12
- License
- NOASSERTION
- Issues
- 450 open
5,557
stars
- Forks
- 485
- Commits
- 6,569 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
slothflowlabs/duckle
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- Updated
- 2026-10-14
- Lists
- 3 list mentions
- First commit
- 2026-05-21
- License
- Apache-2.0
- Issues
- 73 open
1,508
stars
- Forks
- 120
- Commits
- 1,751 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dqops/dqo
Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.
Stack
- Updated
- 2026-01-05
- Lists
- 1 list mention
- First commit
- 2022-03-08
- License
- NOASSERTION
- Issues
- 9 open
194
stars
- Forks
- 38
- Commits
- 11,669 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
vertti/daffy
Lightweight DataFrame validation decorators for Pandas, Polars, Modin, and PyArrow. No custom types required.
AI dev
- Updated
- 2026-10-15
- Lists
- 1 list mention
- First commit
- 2021-01-31
- License
- MIT
- Issues
- 5 open
60
stars
- Forks
- 5
- Commits
- 1,121 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
provero-org/provero
Declarative data quality engine. Define checks in YAML, run anywhere.
AI dev
- Updated
- 2026-10-12
- Lists
- 1 list mention
- First commit
- 2026-03-12
- License
- Apache-2.0
- Issues
- 11 open
17
stars
- Forks
- 3
- Commits
- 133 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rbmuller/scherlok
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
GitHub topics
- Updated
- 2026-09-28
- Lists
- 3 list mentions
- First commit
- 2026-04-01
- License
- MIT
- Issues
- 13 open
11
stars
- Forks
- 6
- Commits
- 118 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
godalida/koala-diff
Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.
GitHub topics
- Updated
- 2026-02-25
- Lists
- 1 list mention
- First commit
- 2026-02-13
- License
- MIT
- Issues
- 0 open
- Forks
- 0
- Commits
- 33 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
aegis-dq/aegis-dq
Open, audit-grade agentic data quality framework with portable industry packs
GitHub topics
- Updated
- 2026-05-27
- Lists
- 2 list mentions
- First commit
- 2026-05-11
- License
- NOASSERTION
- Issues
- 10 open
- Forks
- 2
- Commits
- 76 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%