← All awesome lists Search these repos

Awesome List

Awesome Data Engineering

A curated list of data engineering tools for software developers

igorbarinov/awesome-data-engineering #awesome #awesome-list

Open GitHub

List stars: 9,173
README repos: 148
Indexed repos: 142
List commits: 438
Forks: 1,651
Open issues: 49

Tracked list growth

GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.

Latest scan 2026-10-17 10:49

Likes history

GitHub stars

Commits history

Default branch commits

Indexed repositories

20 repos matching these filters.

Package manager: maven

Latest repo push 2026-10-17

Browse

Filter this list

Search within Awesome Data Engineering or narrow by ecosystem and project health.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

protocolbuffers/protobuf

Protocol Buffers - Google's data interchange format

Stack

C++ Bundler Cargo CMake

GitHub topics

#marshalling #protobuf #protobuf-runtime #protoc #protocol-buffers #protocol-compiler

Updated: 2026-10-17
Lists: 1 list mention
First commit: 2008-07-10
License: NOASSERTION
Issues: 369 open

72,115

stars

Forks: 16,314
Commits: 24,430 commits
Star growth, last 7 days: +12 0.0%
Commit velocity, last 7 days: +104 +0.4%

Website GitHub

juicedata/juicefs

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

AI dev

Stack

Go Cobra gRPC Go Go modules Maven npm

GitHub topics

#bigdata #cloud-native #distributed-systems #filesystem #go #golang

Updated: 2026-10-16
Lists: 4 list mentions
First commit: 2021-01-08
License: Apache-2.0
Issues: 246 open

14,542

stars

Forks: 1,315
Commits: 4,634 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/druid

Apache Druid: a high performance real-time analytics database.

AI dev

Stack

Java React Maven npm

GitHub topics

#druid

Updated: 2026-10-17
Lists: 4 list mentions
First commit: 2012-10-23
License: Apache-2.0
Issues: 777 open

14,088

stars

Forks: 3,802
Commits: 16,747 commits
Star growth, last 7 days: +26 +0.2%
Commit velocity, last 7 days: +44 +0.3%

Website GitHub

xberg-io/xberg

Polyglot document intelligence with a Rust core: extract text, metadata, images, tables, and structured data from 106 formats across 140 file extensions, plus code intelligence for 371 languages. Fifteen bindings, with CLI, REST API, and MCP server.

AI dev

Stack

Rust Android Astro Axum pytest Bundler Cargo Composer

GitHub topics

#bun #csharp #document-intelligence #elixir #ffi #golang

Updated: 2026-10-16
Lists: 3 list mentions
First commit: 2025-01-31
License: MIT
Issues: 14 open

9,422

stars

Forks: 594
Commits: 10,366 commits
Star growth, last 7 days: +22 +0.2%
Commit velocity, last 7 days: +108 +1.1%

Website GitHub

kreuzberg-dev/kreuzberg

A polyglot document intelligence framework with a Rust core. Extract text, metadata, images, and structured information from PDFs, Office documents, images, and 97+ formats. Available for Rust, Python, Ruby, Java, Go, PHP, Elixir, C#, R, C, TypeScript (Node/Bun/Wasm/Deno)- or use via CLI, REST API, or MCP server.

Stack

Rust Android Axum pytest Shelf Bundler Cargo Composer

GitHub topics

#bun #csharp #document-intelligence #elixir #ffi #golang

Updated: 2026-06-23
Lists: 3 list mentions
First commit: 2025-01-31
License: NOASSERTION
Issues: 13 open

8,536

stars

Forks: 502
Commits: 6,280 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

EsotericSoftware/kryo

Java binary serialization and cloning: fast, efficient, automatic

Stack

HTML Maven

GitHub topics

#cloning #java #kryo #serialization

Updated: 2026-10-15
Lists: 1 list mention
First commit: 2009-09-23
License: BSD-3-Clause
Issues: 5 open

6,545

stars

Forks: 844
Commits: 1,712 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

treeverse/lakeFS

lakeFS - Data version control for your data lake | Git for data

Stack

Go Android Cobra Gin gRPC Go Bundler Cargo Go modules

GitHub topics

#apache-spark #apache-sparksql #aws-s3 #azure-blob-storage #azure-storage #data-engineering

Updated: 2026-10-07
Lists: 2 list mentions
First commit: 2019-09-12
License: NOASSERTION
Issues: 450 open

5,557

stars

Forks: 485
Commits: 6,569 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

pinterest/secor

Secor is a service implementing Kafka log persistence

Archived AI dev

Stack

Java Maven

GitHub topics

#kafka

Updated: 2026-10-04
Lists: 2 list mentions
First commit: 2014-04-15
License: Apache-2.0
Issues: 284 open

1,858

stars

Forks: 536
Commits: 3,340 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

gchq/Gaffer

A large-scale entity and relation database supporting aggregation of properties

Archived

Stack

Java Spring Boot Maven

GitHub topics

#accumulo #aggregation #big-data #graph #graph-database #hadoop

Updated: 2025-06-06
Lists: 2 list mentions
First commit: 2015-12-14
License: Apache-2.0
Issues: 138 open

1,787

stars

Forks: 363
Commits: 7,332 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

kairosdb/kairosdb

Fast scalable time series database

Stack

Java Maven

GitHub topics

#java #kairosdb #metrics #timeseries #timeseries-database

Updated: 2026-03-05
Lists: 3 list mentions
First commit: 2013-02-05
License: Apache-2.0
Issues: 141 open

1,760

stars

Forks: 341
Commits: 1,479 commits
Star growth, last 7 days: -1 -0.1%
Commit velocity, last 7 days: 0 0.0%

GitHub

aklivity/zilla

🦎 A lightweight, multi-protocol gateway for event-driven applications and AI agents. Expose and govern Kafka, MQTT, APIs, and MCP through one high-performance engine with shared routing, security, schema, and observability.

AI dev

Stack

Java Maven

GitHub topics

#api-gateway #asyncapi #event-driven-architecture #event-stream-proxy #event-streaming #grpc

Updated: 2026-10-16
Lists: 2 list mentions
First commit: 2021-12-07
License: NOASSERTION
Issues: 241 open

1,710

stars

Forks: 147
Commits: 4,224 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: +7 +0.2%

Website GitHub

projectnessie/nessie

Nessie: Transactional Catalog for Data Lakes with Git-like semantics

AI dev

Stack

Java Quarkus Gradle Maven pip

GitHub topics

#aws-lambda #data #git #iceberg #java #spark

Updated: 2026-10-16
Lists: 1 list mention
First commit: 2020-04-07
License: Apache-2.0
Issues: 169 open

1,520

stars

Forks: 198
Commits: 11,882 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

rax-maas/blueflood

A distributed system designed to ingest and process time series data

Stack

Java Maven npm pip

Updated: 2024-08-19
Lists: 2 list mentions
First commit: 2013-08-19
License: Apache-2.0
Issues: 54 open

598

stars

Forks: 102
Commits: 3,152 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

NationalSecurityAgency/timely

Accumulo backed time series database

Stack

Java gRPC Go React Spring Boot Go modules Maven npm

GitHub topics

#accumulo #hacktoberfest #series-database #time-series #timeseries

Updated: 2026-10-02
Lists: 2 list mentions
First commit: 2016-04-21
License: Apache-2.0
Issues: 20 open

401

stars

Forks: 109
Commits: 490 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

kroxylicious/kroxylicious

Kroxylicious, the snappy open source proxy for Apache Kafka®

AI dev

Stack

Java Maven pip

GitHub topics

#apache-kafka #java #kafka #proxy

Updated: 2026-10-16
Lists: 1 list mention
First commit: 2021-09-14
License: Apache-2.0
Issues: 377 open

330

stars

Forks: 136
Commits: 5,930 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/incubator-hivemall

Mirror of Apache Hivemall (incubating)

Archived

Stack

Java Maven

GitHub topics

#hivemall

Updated: 2022-09-06
Lists: 1 list mention
First commit: 2013-09-26
License: Apache-2.0
Issues: Disabled

312

stars

Forks: 112
Commits: 2,037 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

Stratio/deep-spark

Connecting Apache Spark with different data stores [DEPRECATED]

Archived

Stack

Java Maven

Updated: 2016-07-04
Lists: 1 list mention
First commit: 2014-02-18
License: Apache-2.0
Issues: 0 open

197

stars

Forks: 43
Commits: 1,084 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

dqops/dqo

Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.

Stack

Java Flask React Spring Boot Tailwind CSS Maven npm PEP 517

GitHub topics

#data-observability #data-ops #data-profiling #data-quality #data-quality-checks #data-quality-measurement

Updated: 2026-01-05
Lists: 1 list mention
First commit: 2022-03-08
License: NOASSERTION
Issues: 9 open

194

stars

Forks: 38
Commits: 11,669 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

scriptella/scriptella-etl

Scriptella is an open source ETL (Extract-Transform-Load) and script execution tool written in Java.

Stack

Java Maven

GitHub topics

#csv #data-engineering #data-integration #data-migration #etl #java

Updated: 2026-10-15
Lists: 1 list mention
First commit: 2006-04-17
License: Apache-2.0
Issues: 18 open

126

stars

Forks: 50
Commits: 895 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

chrislusf/seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

Stack

Go Cobra Gin gRPC Go Cargo Go modules Maven

Updated: 2026-10-16
Lists: 3 list mentions
First commit: 2011-11-30
License: Apache-2.0
Issues: Disabled

stars

Forks: 7
Commits: 12,815 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Activity

Default branch: master
Last pushed: 2026-09-07
GitHub updated: 2026-10-17
Created: 2015-06-18
First commit: 2015-06-18
Last scanned: 2026-10-17 10:49
Watchers: 252

Indexed repo mix

Repo stars: 875,040
Repo forks: 139,169
Active: 120
Archived: 22

Languages

Python (39) Go (21) Java (20) C++ (10) JavaScript (9) Rust (9) Scala (7) C (5) TypeScript (5) Shell (3) Kotlin (2) Ruby (2)

Awesome Data Engineering

Tracked list growth

Likes history

Commits history

Indexed repositories

Filter this list

Put your repository first

How it works

Pricing