Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
Awesome Data Engineering
A curated list of data engineering tools for software developers
igorbarinov/awesome-data-engineering
#awesome
#awesome-list
- List stars
- 9,173
- README repos
- 148
- Indexed repos
- 142
- List commits
- 438
- Forks
- 1,651
- Open issues
- 49
Tracked list growth
GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.
Latest scan 2026-10-17 10:49
Likes history
GitHub starsCommits history
Default branch commits
Latest repo push 2026-10-17
bruin-data/ingestr
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 3 list mentions
- First commit
- 2024-02-12
- License
- NOASSERTION
- Issues
- 28 open
4,013
stars
- Forks
- 163
- Commits
- 3,719 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
slothflowlabs/duckle
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- Updated
- 2026-10-14
- Lists
- 3 list mentions
- First commit
- 2026-05-21
- License
- Apache-2.0
- Issues
- 73 open
1,508
stars
- Forks
- 120
- Commits
- 1,751 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
SouravRoy-ETL/slothdb
An experimental embedded SQL engine in C++20. Query Parquet, CSV, JSON, Arrow, Avro, SQLite, and Excel files directly with SQL, in-process. Early-stage.
GitHub topics
- Updated
- 2026-06-03
- Lists
- 1 list mention
- First commit
- 2026-04-15
- License
- MIT
- Issues
- 5 open
412
stars
- Forks
- 6
- Commits
- 573 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
ralforion/orionbelt-semantic-layer
Source-available semantic and context layer, rule engine, and semantic sidecar for agentic AI and governed analytics. Compiles declarative YAML models into optimized SQL, KPIs, and semantic context across 8 SQL dialects.
AI dev
- Updated
- 2026-10-16
- Lists
- 2 list mentions
- First commit
- 2026-02-16
- License
- NOASSERTION
- Issues
- 1 open
- Forks
- 11
- Commits
- 742 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
nnnkkk7/snowflake-emulator
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
Stack
- Updated
- 2026-08-17
- Lists
- 2 list mentions
- First commit
- 2025-12-29
- License
- MIT
- Issues
- 10 open
- Forks
- 14
- Commits
- 43 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
MoonMao42/ReceiptBI
Turn CSV, Excel, and read-only databases into verified, editable business reports.
- Updated
- 2026-10-16
- Lists
- 1 list mention
- First commit
- 2025-12-12
- License
- MIT
- Issues
- 0 open
39
stars
- Forks
- 0
- Commits
- 316 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
drt-hub/drt
Reverse ETL for the code-first data stack
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 1 list mention
- First commit
- 2026-03-27
- License
- Apache-2.0
- Issues
- 53 open
36
stars
- Forks
- 39
- Commits
- 832 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
provero-org/provero
Declarative data quality engine. Define checks in YAML, run anywhere.
AI dev
- Updated
- 2026-10-12
- Lists
- 1 list mention
- First commit
- 2026-03-12
- License
- Apache-2.0
- Issues
- 11 open
17
stars
- Forks
- 3
- Commits
- 133 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rbmuller/scherlok
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
GitHub topics
- Updated
- 2026-09-28
- Lists
- 3 list mentions
- First commit
- 2026-04-01
- License
- MIT
- Issues
- 13 open
11
stars
- Forks
- 6
- Commits
- 118 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
ImpossibleForge/pfc-jsonl
Cold storage that queries like hot storage. High-ratio JSONL compressor with block-level random access. Query cold storage in seconds — no restore, no full decompression. 5–13% ratio on JSONL logs, 25–57% smaller than gzip. 1-hour window via DuckDB in under 6s. Free for personal and open-source use.
GitHub topics
- Updated
- 2026-06-19
- Lists
- 2 list mentions
- First commit
- 2026-04-01
- License
- NOASSERTION
- Issues
- 0 open
- Forks
- 0
- Commits
- 58 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
aegis-dq/aegis-dq
Open, audit-grade agentic data quality framework with portable industry packs
GitHub topics
- Updated
- 2026-05-27
- Lists
- 2 list mentions
- First commit
- 2026-05-11
- License
- NOASSERTION
- Issues
- 10 open
- Forks
- 2
- Commits
- 76 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
dataspoclab/dataspoc-pipe
Data ingestion engine — Singer taps to Parquet in cloud buckets
AI dev
GitHub topics
- Updated
- 2026-04-28
- Lists
- 1 list mention
- First commit
- 2026-03-24
- License
- Apache-2.0
- Issues
- 0 open
- Forks
- 0
- Commits
- 5 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%