Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 21,464
Awesome lists tracked: 149
Current results: 18

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Unsure which signals matter? Read the repository shortlisting guide.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

18 repos shown

Topic: data-pipelines

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

pathwaycom/pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Stack

Python Axum FastAPI LangChain pytest Cargo npm PEP 517

GitHub topics

#batch-processing #data-analytics #data-pipelines #data-processing #dataflow #etl

Updated: 2026-10-16
Lists: 5 list mentions
First commit: 2023-07-21
History: 11 history points
License: NOASSERTION
Issues: 37 open

62,113

stars

Forks: 1,683
Commits: 2,312 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

AI dev

Stack

Python Celery Cobra FastAPI Flask Go modules Gradle npm

GitHub topics

#airflow #apache #apache-airflow #automation #dag #data-engineering

Updated: 2026-10-17
Lists: 7 list mentions
First commit: 2014-10-06
History: 154 history points
License: Apache-2.0
Issues: 1,932 open

47,194

stars

Forks: 18,051
Commits: 42,128 commits
Star growth, last 7 days: +49 +0.1%
Commit velocity, last 7 days: +168 +0.4%

Website GitHub

dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

AI dev

Stack

Python FastAPI Flask LangChain Next.js npm PEP 517 pip

GitHub topics

#analytics #dagster #data-engineering #data-integration #data-orchestrator #data-pipelines

Updated: 2026-10-12
Lists: 5 list mentions
First commit: 2018-04-30
History: 13 history points
License: Apache-2.0
Issues: 2,590 open

16,282

stars

Forks: 2,354
Commits: 28,117 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Unstructured-IO/unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

Stack

HTML pytest pip uv

GitHub topics

#data-pipelines #deep-learning #document-image-analysis #document-image-processing #document-parser #document-parsing

Updated: 2026-09-10
Lists: 2 list mentions
First commit: 2022-06-29
History: 1 history point
License: Apache-2.0
Issues: 306 open

15,412

stars

Forks: 1,323
Commits: 1,928 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

apache/dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

AI dev

Stack

Java Spring Boot Vite Vue Maven npm pnpm

GitHub topics

#airflow #azkaban #cloud-native #data-pipelines #job-scheduler #orchestration

Updated: 2026-09-15
Lists: 1 list mention
First commit: 2019-03-01
History: 6 history points
License: Apache-2.0
Issues: 132 open

14,475

stars

Forks: 5,092
Commits: 9,072 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

kedro-org/kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

AI dev

Stack

Python FastAPI Jupyter pytest PEP 517 pip

GitHub topics

#agentic-ai #agentic-workflow #data-pipelines #hacktoberfest #kedro #machine-learning

Updated: 2026-09-09
Lists: 1 list mention
First commit: 2019-05-16
History: 7 history points
License: NOASSERTION
Issues: 138 open

10,989

stars

Forks: 1,074
Commits: 4,399 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

mage-ai/mage-ai

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Stack

Python LangChain Next.js pytest React npm PEP 517 pip

GitHub topics

#artificial-intelligence #data #data-engineering #data-integration #data-pipelines #data-science

Updated: 2026-09-10
Lists: 1 list mention
First commit: 2022-05-16
History: 7 history points
License: Apache-2.0
Issues: 624 open

8,821

stars

Forks: 990
Commits: 5,790 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

OpenDCAI/DataFlow

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

Stack

Python FastAPI Flask Gradio pytest PEP 517 pip

GitHub topics

#data #data-agent #data-cleaning #data-pipelines #data-processing #data-science

Updated: 2026-10-13
Lists: 3 list mentions
First commit: 2024-10-13
History: 2 history points
License: Apache-2.0
Issues: 24 open

8,209

stars

Forks: 1,196
Commits: 1,209 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

fluvio-community/fluvio

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

Stack

Rust Cargo

GitHub topics

#cloud-native #data-analytics #data-flow #data-integration #data-pipelines #distributed-systems

Updated: 2026-08-30
Lists: 2 list mentions
First commit: 2019-09-04
History: 5 history points
License: Apache-2.0
Issues: 139 open

5,256

stars

Forks: 531
Commits: 2,574 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

StructuredLabs/preswald

Preswald is a WASM packager for Python-based interactive data apps: bundle full complex data workflows, particularly visualizations, into single files, runnable completely in-browser, using Pyodide, DuckDB, Pandas, and Plotly, Matplotlib, etc. Build dashboards, reports, and notebooks that run offline, load fast, and share like a document.

Stack

Python FastAPI pytest React Tailwind CSS npm PEP 517

GitHub topics

#ai #analytics #analytics-engineering #copilot #data #data-applications

Updated: 2026-06-11
Lists: 2 list mentions
First commit: 2024-12-14
History: 2 history points
License: Apache-2.0
Issues: 330 open

4,274

stars

Forks: 647
Commits: 546 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

ucbepic/docetl

A system for agentic LLM-powered data processing and ETL

AI dev

Stack

Python FastAPI Next.js pytest React npm pip uv

GitHub topics

#agents #data #data-pipelines #document-analysis #document-processing #elt

Updated: 2026-10-09
Lists: 2 list mentions
First commit: 2024-07-09
History: 139 history points
License: MIT
Issues: 45 open

4,133

stars

Forks: 450
Commits: 920 commits
Star growth, last 7 days: +5 +0.1%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

orchest/orchest

Build data pipelines, the easy way 🛠️

Archived

Stack

TypeScript Celery Cobra Flask gRPC Go Go modules npm PEP 517

GitHub topics

#airflow #cloud #dag #data-pipelines #data-science #deployment

Updated: 2023-06-06
Lists: 1 list mention
First commit: 2019-11-28
History: 5 history points
License: Apache-2.0
Issues: 130 open

4,131

stars

Forks: 258
Commits: 9,688 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Netflix/maestro

Maestro: Netflix’s Workflow Orchestrator

Stack

Java Spring Boot Gradle

GitHub topics

#agentic-workflow #analytics #automation #batch-processing #dag #data-engineering

Updated: 2026-09-03
Lists: 1 list mention
First commit: 2024-04-17
History: 7 history points
License: Apache-2.0
Issues: 58 open

3,834

stars

Forks: 311
Commits: 173 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

meltano/meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

AI dev

Stack

Python pytest React Tailwind CSS npm PEP 517 pip

GitHub topics

#connectors #data #data-engineering #data-pipelines #dataops #dataops-platform

Updated: 2026-10-13
Lists: 1 list mention
First commit: 2018-04-27
History: 138 history points
License: MIT
Issues: 145 open

2,659

stars

Forks: 273
Commits: 13,065 commits
Star growth, last 7 days: +7 +0.3%
Commit velocity, last 7 days: +12 +0.1%

Website GitHub

bruin-data/bruin

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

AI dev

Stack

Go Cobra Flask gRPC Go Vite Go modules npm PEP 517

GitHub topics

#analytics #bigquery #data-analysis #data-ingestion #data-modeling #data-pipelines

Updated: 2026-10-15
Lists: 3 list mentions
First commit: 2023-07-25
History: 45 history points
License: Apache-2.0
Issues: 66 open

1,779

stars

Forks: 98
Commits: 8,629 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

pyper-dev/pyper

Concurrent Python made simple

Stack

Python pytest Bundler PEP 517 pip

GitHub topics

#asyncio #concurrency #data #data-collection #data-engineering #data-pipelines

Updated: 2025-02-04
Lists: 0 list mentions
First commit: 2024-05-22
History: 42 history points
License: MIT
Issues: 5 open

1,517

stars

Forks: 32
Commits: 115 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

amphi-ai/amphi-etl

visual data prep powered by python

AI dev

Stack

TypeScript pytest React Tornado npm PEP 517 pip

GitHub topics

#analytics-automation #data #data-analysis #data-pipelines #data-preparation #data-science

Updated: 2026-08-24
Lists: 1 list mention
First commit: 2024-03-21
History: 136 history points
License: NOASSERTION
Issues: 127 open

1,409

stars

Forks: 108
Commits: 549 commits
Star growth, last 7 days: -2 -0.1%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Burla-Cloud/burla

The world's simplest distributed computing framework.

AI dev

Stack

Python FastAPI pytest React Starlette Bun npm uv

GitHub topics

#batch-processing #data-pipelines #etl-pipelines #python

Updated: 2026-10-10
Lists: 0 list mentions
First commit: 2024-03-19
History: 136 history points
License: NOASSERTION
Issues: 2 open

263

stars

Forks: 11
Commits: 909 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing