Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
A curated list of data engineering tools for software developers
GitHub stars and default-branch commits for igorbarinov/awesome-data-engineering.
Open highlighted repo slot
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Scalable datastore for metrics, events, and real-time analytics
Incremental engine for long horizon agents 🌟 Star if you like it!
Polyglot document intelligence with a Rust core: extract text, metadata, images, tables, and structured data from 106 formats across 140 file extensions, plus code intelligence for 371 languages. Fifteen bindings, with CLI, REST API, and MCP server.
A polyglot document intelligence framework with a Rust core. Extract text, metadata, images, and structured information from PDFs, Office documents, images, and 97+ formats. Available for Rust, Python, Ruby, Java, Go, PHP, Elixir, C#, R, C, TypeScript (Node/Bun/Wasm/Deno)- or use via CLI, REST API, or MCP server.
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
Object-storage native graph database with branching and typed ontology
High-performance, time-indexed object storage for robotics and industrial IoT
The fast, config-driven way to move data in Rust — a native CLI and an embeddable Rust ETL library