Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
Awesome Data Engineering
A curated list of data engineering tools for software developers
igorbarinov/awesome-data-engineering
#awesome
#awesome-list
- List stars
- 9,173
- README repos
- 148
- Indexed repos
- 142
- List commits
- 438
- Forks
- 1,651
- Open issues
- 49
Tracked list growth
GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.
Latest scan 2026-10-17 10:49
Likes history
GitHub starsCommits history
Default branch commits
Latest repo push 2026-10-17
prometheus/prometheus
The Prometheus monitoring system and time series database.
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 6 list mentions
- First commit
- 2012-11-24
- License
- Apache-2.0
- Issues
- 983 open
66,598
stars
- Forks
- 10,917
- Commits
- 18,958 commits
- Star growth, last 7 days
- +136 +0.2%
- Commit velocity, last 7 days
- +117 +0.6%
pingcap/tidb
TiDB is built for agentic workloads that grow unpredictably, with ACID guarantees and native support for transactions, analytics, and vector search. No data silos. No noisy neighbors. No infrastructure ceiling.
AI dev
Stack
GitHub topics
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2015-09-06
- License
- Apache-2.0
- Issues
- 7,367 open
40,645
stars
- Forks
- 6,266
- Commits
- 28,736 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
google/cadvisor
Analyzes resource usage and performance characteristics of running containers.
Stack
- Updated
- 2026-10-12
- Lists
- 5 list mentions
- First commit
- 2014-06-09
- License
- NOASSERTION
- Issues
- 72 open
19,481
stars
- Forks
- 2,496
- Commits
- 3,782 commits
- Star growth, last 7 days
- +7 0.0%
- Commit velocity, last 7 days
- +1 0.0%
rqlite/rqlite
The lightweight, fault-tolerant database built on SQLite. Designed to keep your data highly available with minimal effort.
AI dev
Stack
- Updated
- 2026-10-15
- Lists
- 4 list mentions
- First commit
- 2014-08-23
- License
- MIT
- Issues
- 65 open
17,794
stars
- Forks
- 818
- Commits
- 8,689 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
hashicorp/nomad
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.
- Updated
- 2026-10-15
- Lists
- 3 list mentions
- First commit
- 2015-06-01
- License
- NOASSERTION
- Issues
- 1,626 open
17,011
stars
- Forks
- 2,148
- Commits
- 28,321 commits
- Star growth, last 7 days
- +10 +0.1%
- Commit velocity, last 7 days
- +5 0.0%
cayleygraph/cayley
An open-source graph database
Stack
GitHub topics
- Updated
- 2026-08-27
- Lists
- 2 list mentions
- First commit
- 2014-06-20
- License
- Apache-2.0
- Issues
- 93 open
15,063
stars
- Forks
- 1,234
- Commits
- 1,776 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
juicedata/juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2021-01-08
- License
- Apache-2.0
- Issues
- 246 open
14,542
stars
- Forks
- 1,315
- Commits
- 4,634 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
weaveworks/weave
Simple, resilient multi-host containers networking and more.
Archived
Stack
GitHub topics
- Updated
- 2024-08-09
- Lists
- 1 list mention
- First commit
- 2014-08-18
- License
- Apache-2.0
- Issues
- 529 open
6,609
stars
- Forks
- 672
- Commits
- 7,293 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
treeverse/lakeFS
lakeFS - Data version control for your data lake | Git for data
- Updated
- 2026-10-07
- Lists
- 2 list mentions
- First commit
- 2019-09-12
- License
- NOASSERTION
- Issues
- 450 open
5,557
stars
- Forks
- 485
- Commits
- 6,569 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rudderlabs/rudder-server
Privacy and Security focused Segment-alternative, in Golang and React
AI dev
Stack
- Updated
- 2026-10-16
- Lists
- 4 list mentions
- First commit
- 2019-07-19
- License
- NOASSERTION
- Issues
- 50 open
4,500
stars
- Forks
- 80
- Commits
- 6,733 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
bruin-data/ingestr
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 3 list mentions
- First commit
- 2024-02-12
- License
- NOASSERTION
- Issues
- 28 open
4,013
stars
- Forks
- 163
- Commits
- 3,719 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
mozilla-services/heka
DEPRECATED: Data collection and processing made easy.
Archived
- Updated
- 2024-01-23
- Lists
- 2 list mentions
- First commit
- 2012-10-10
- License
- NOASSERTION
- Issues
- 232 open
3,387
stars
- Forks
- 514
- Commits
- 4,118 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
bruin-data/bruin
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
AI dev
- Updated
- 2026-10-15
- Lists
- 3 list mentions
- First commit
- 2023-07-25
- License
- Apache-2.0
- Issues
- 66 open
1,779
stars
- Forks
- 98
- Commits
- 8,629 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
lf-edge/ekuiper
Lightweight data stream processing engine for IoT edge
Stack
GitHub topics
- Updated
- 2026-10-16
- Lists
- 2 list mentions
- First commit
- 2019-07-03
- License
- Apache-2.0
- Issues
- 47 open
1,739
stars
- Forks
- 464
- Commits
- 3,719 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
prometheus/haproxy_exporter
Simple server that scrapes HAProxy stats and exports them via HTTP for Prometheus consumption
Archived
Stack
GitHub topics
- Updated
- 2023-03-08
- Lists
- 1 list mention
- First commit
- 2013-01-31
- License
- Apache-2.0
- Issues
- 0 open
- Forks
- 217
- Commits
- 293 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
brexhq/substation
Substation is a toolkit for routing, normalizing, and enriching security event and audit logs.
Stack
GitHub topics
- Updated
- 2026-01-20
- Lists
- 3 list mentions
- First commit
- 2022-04-19
- License
- MIT
- Issues
- 3 open
408
stars
- Forks
- 35
- Commits
- 250 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
grammarly/rocker-compose
Docker composition tool with idempotency features for deploying apps composed of multiple containers.
Archived
Stack
- Updated
- 2023-03-24
- Lists
- 1 list mention
- First commit
- 2015-06-24
- License
- NOASSERTION
- Issues
- 0 open
408
stars
- Forks
- 20
- Commits
- 524 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
CenturyLinkLabs/zodiac
A lightweight tool for easy deployment and rollback of dockerized applications.
Stack
- Updated
- 2020-01-24
- Lists
- 1 list mention
- First commit
- 2015-05-05
- License
- Apache-2.0
- Issues
- 2 open
200
stars
- Forks
- 20
- Commits
- 72 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
nnnkkk7/snowflake-emulator
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
Stack
- Updated
- 2026-08-17
- Lists
- 2 list mentions
- First commit
- 2025-12-29
- License
- MIT
- Issues
- 10 open
- Forks
- 14
- Commits
- 43 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
chrislusf/seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.
- Updated
- 2026-10-16
- Lists
- 3 list mentions
- First commit
- 2011-11-30
- License
- Apache-2.0
- Issues
- Disabled
43
stars
- Forks
- 7
- Commits
- 12,815 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
rbmuller/datatrax
Data engineering and machine learning toolkit for Go.
Stack
- Updated
- 2026-05-01
- Lists
- 2 list mentions
- First commit
- 2023-11-23
- License
- MIT
- Issues
- 13 open
10
stars
- Forks
- 0
- Commits
- 19 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%