← All awesome lists Search these repos

Awesome List

Awesome Data Engineering

A curated list of data engineering tools for software developers

igorbarinov/awesome-data-engineering #awesome #awesome-list

Open GitHub

List stars: 9,173
README repos: 148
Indexed repos: 142
List commits: 438
Forks: 1,651
Open issues: 49

Tracked list growth

GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.

Latest scan 2026-10-17 10:49

Likes history

GitHub stars

Commits history

Default branch commits

Indexed repositories

14 repos matching these filters.

Package manager: gradle

Latest repo push 2026-10-17

Browse

Filter this list

Search within Awesome Data Engineering or narrow by ecosystem and project health.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

AI dev

Stack

Python Celery Cobra FastAPI Flask Go modules Gradle npm

GitHub topics

#airflow #apache #apache-airflow #automation #dag #data-engineering

Updated: 2026-10-17
Lists: 7 list mentions
First commit: 2014-10-06
License: Apache-2.0
Issues: 1,932 open

47,194

stars

Forks: 18,051
Commits: 42,128 commits
Star growth, last 7 days: +49 +0.1%
Commit velocity, last 7 days: +168 +0.4%

Website GitHub

kestra-io/kestra

Event Driven Orchestration & Scheduling Platform for Mission Critical Applications

AI dev

Stack

Java Micronaut React Vite Vue Gradle npm

GitHub topics

#ai-agents #automation #control-plane #data-engineering #data-orchestration #data-orchestrator

Updated: 2026-10-16
Lists: 4 list mentions
First commit: 2019-08-13
License: Apache-2.0
Issues: 888 open

29,650

stars

Forks: 3,405
Commits: 13,021 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

xberg-io/xberg

Polyglot document intelligence with a Rust core: extract text, metadata, images, tables, and structured data from 106 formats across 140 file extensions, plus code intelligence for 371 languages. Fifteen bindings, with CLI, REST API, and MCP server.

AI dev

Stack

Rust Android Astro Axum pytest Bundler Cargo Composer

GitHub topics

#bun #csharp #document-intelligence #elixir #ffi #golang

Updated: 2026-10-16
Lists: 3 list mentions
First commit: 2025-01-31
License: MIT
Issues: 14 open

9,422

stars

Forks: 594
Commits: 10,366 commits
Star growth, last 7 days: +22 +0.2%
Commit velocity, last 7 days: +108 +1.1%

Website GitHub

kreuzberg-dev/kreuzberg

A polyglot document intelligence framework with a Rust core. Extract text, metadata, images, and structured information from PDFs, Office documents, images, and 97+ formats. Available for Rust, Python, Ruby, Java, Go, PHP, Elixir, C#, R, C, TypeScript (Node/Bun/Wasm/Deno)- or use via CLI, REST API, or MCP server.

Stack

Rust Android Axum pytest Shelf Bundler Cargo Composer

GitHub topics

#bun #csharp #document-intelligence #elixir #ffi #golang

Updated: 2026-06-23
Lists: 3 list mentions
First commit: 2025-01-31
License: NOASSERTION
Issues: 13 open

8,536

stars

Forks: 502
Commits: 6,280 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

treeverse/lakeFS

lakeFS - Data version control for your data lake | Git for data

Stack

Go Android Cobra Gin gRPC Go Bundler Cargo Go modules

GitHub topics

#apache-spark #apache-sparksql #aws-s3 #azure-blob-storage #azure-storage #data-engineering

Updated: 2026-10-07
Lists: 2 list mentions
First commit: 2019-09-12
License: NOASSERTION
Issues: 450 open

5,557

stars

Forks: 485
Commits: 6,569 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

AI dev

Stack

Java pytest Starlette Cargo Gradle PEP 517

GitHub topics

#ai-catalog #data-catalog #datalake #federated-query #lakehouse #metadata

Updated: 2026-10-16
Lists: 1 list mention
First commit: 2023-04-23
License: Apache-2.0
Issues: 1,099 open

3,249

stars

Forks: 957
Commits: 4,671 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

jprante/elasticsearch-jdbc

JDBC importer for Elasticsearch *** THIS REPOSITORY WILL BE DELETED WITHOUT ANY FURTHER NOTICE AFTER AUG 1 2026 ***

Archived

Stack

Java Gradle

Updated: 2026-06-03
Lists: 1 list mention
First commit: 2012-06-02
License: Apache-2.0
Issues: 1 open

2,805

stars

Forks: 693
Commits: 453 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

apache/gobblin

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

Stack

Java Gradle pip

GitHub topics

#apache #data #ingestion #management #replication

Updated: 2026-10-15
Lists: 2 list mentions
First commit: 2014-02-04
License: Apache-2.0
Issues: Disabled

2,270

stars

Forks: 749
Commits: 6,560 commits
Star growth, last 7 days: +1 0.0%
Commit velocity, last 7 days: +1 0.0%

Website GitHub

projectnessie/nessie

Nessie: Transactional Catalog for Data Lakes with Git-like semantics

AI dev

Stack

Java Quarkus Gradle Maven pip

GitHub topics

#aws-lambda #data #git #iceberg #java #spark

Updated: 2026-10-16
Lists: 1 list mention
First commit: 2020-04-07
License: Apache-2.0
Issues: 169 open

1,520

stars

Forks: 198
Commits: 11,882 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

TIBCOSoftware/snappydata

Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster

Stack

Scala Gradle pip

GitHub topics

#analytics #memory-database #scale #snappydata #spark #stream

Updated: 2022-11-21
Lists: 2 list mentions
First commit: 2015-05-13
License: NOASSERTION
Issues: 117 open

1,033

stars

Forks: 197
Commits: 4,151 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

spotify/heroic

The Heroic Time Series Database

Archived

Stack

Java Bundler Gradle pip

GitHub topics

#cassandra #elasticsearch #google-bigtable #google-pubsub #ha #java

Updated: 2021-03-26
Lists: 1 list mention
First commit: 2015-06-03
License: Apache-2.0
Issues: 70 open

845

stars

Forks: 105
Commits: 1,296 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

kakao/actionbase

One database for user-user (follows), user-item (likes), and item-item (related items) interactions — precomputed at write time, served as simple lookups

Stack

Kotlin Astro React Vite Go modules Gradle npm

GitHub topics

#database #graphdb #high-throughput #low-latency #oltp #real-time

Updated: 2026-10-14
Lists: 2 list mentions
First commit: 2025-12-09
License: Apache-2.0
Issues: 0 open

227

stars

Forks: 18
Commits: 323 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

getstrm/pace

Data policy IN, dynamic view OUT: PACE is the Policy As Code Engine. It helps you to programatically create and apply a data policy to a processing platform like Databricks, Snowflake or BigQuery (or plain 'ol Postgres, even!) with definitions imported from Collibra, Datahub, ODD and the like.

Stack

Kotlin Ktor Spring Boot Gradle npm

GitHub topics

#bigquery #data-catalog #data-contracts #data-governance #data-processing #databricks

Updated: 2026-10-15
Lists: 1 list mention
First commit: 2023-10-18
License: Apache-2.0
Issues: 16 open

stars

Forks: 1
Commits: 635 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

asavinov/bistro

A general-purpose data analysis engine radically changing the way batch and stream data is processed

Stack

Java Gradle

GitHub topics

#analytics #big-data-analytics #edge-analytics #iot #stream-analytics #stream-processing

Updated: 2018-09-07
Lists: 1 list mention
First commit: 2017-09-03
License: MIT
Issues: 1 open

stars

Forks: 0
Commits: 141 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Activity

Default branch: master
Last pushed: 2026-09-07
GitHub updated: 2026-10-17
Created: 2015-06-18
First commit: 2015-06-18
Last scanned: 2026-10-17 10:49
Watchers: 252

Indexed repo mix

Repo stars: 875,040
Repo forks: 139,169
Active: 120
Archived: 22

Languages

Python (39) Go (21) Java (20) C++ (10) JavaScript (9) Rust (9) Scala (7) C (5) TypeScript (5) Shell (3) Kotlin (2) Ruby (2)

Awesome Data Engineering

Tracked list growth

Likes history

Commits history

Indexed repositories

Filter this list

Put your repository first

How it works

Pricing