Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 21,464
Awesome lists tracked: 149
Current results: 49

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Unsure which signals matter? Read the repository shortlisting guide.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

49 repos shown

Topic: inference

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

superlinked/sie

Open-source inference server and production cluster for all the models your agent needs.

AI dev

Stack

Python Axum Express FastAPI Flask Bun Cargo npm

GitHub topics

#bge #colbert #data-pipeline #deep-learning #embeddings #inference

Updated: 2026-09-10
Lists: 1 list mention
First commit: 2026-04-01
History: 1 history point
License: Apache-2.0
Issues: 16 open

3,258

stars

Forks: 307
Commits: 155 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

pytorch/ao

PyTorch native quantization for training and inference

AI dev

Stack

Python CMake PEP 517 pip

GitHub topics

#brrr #cuda #dtypes #float8 #inference #llama

Updated: 2026-10-16
Lists: 1 list mention
First commit: 2023-11-03
History: 139 history points
License: NOASSERTION
Issues: 811 open

3,005

stars

Forks: 653
Commits: 2,827 commits
Star growth, last 7 days: +6 +0.2%
Commit velocity, last 7 days: +8 +0.3%

Website GitHub

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

AI dev

Stack

C++ FastAPI pytest CMake PEP 517 pip

GitHub topics

#ascend #inference #llm #llm-serving #llmops #mlops

Updated: 2026-09-11
Lists: 2 list mentions
First commit: 2025-01-29
History: 1 history point
License: Apache-2.0
Issues: 3,178 open

2,800

stars

Forks: 2,234
Commits: 5,179 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

roboflow/inference

Turn any computer or edge device into a command center for your computer vision projects.

AI dev

Stack

Python Celery FastAPI Flask Next.js npm PEP 517 pip

GitHub topics

#agents #classification #computer-vision #deployment #docker #inference

Updated: 2026-10-14
Lists: 1 list mention
First commit: 2023-07-31
History: 3 history points
License: NOASSERTION
Issues: 187 open

2,477

stars

Forks: 330
Commits: 11,675 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

adithya-s-k/AI-Engineering.academy

Mastering Applied AI, One Concept at a Time

Stack

Jupyter Notebook FastAPI LangChain LlamaIndex React npm pip

GitHub topics

#fine-tuning #finetuning #finetuning-llms #inference #large-language-models #llm

Updated: 2026-02-27
Lists: 1 list mention
First commit: 2023-10-05
History: 2 history points
License: MIT
Issues: 9 open

2,383

stars

Forks: 286
Commits: 429 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

dstackai/dstack

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

AI dev

Stack

Python FastAPI gRPC Go pytest React Go modules npm PEP 517

GitHub topics

#agent-skills #agentic-orchestration #amd #cloud #containers #docker

Updated: 2026-09-11
Lists: 1 list mention
First commit: 2022-01-13
History: 7 history points
License: MPL-2.0
Issues: 69 open

2,244

stars

Forks: 261
Commits: 3,852 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

envoyproxy/ai-gateway

Manages Unified Access to Generative AI Services built on Envoy Gateway

AI dev

Stack

Go Cobra FastAPI gRPC Go React Go modules npm pip

GitHub topics

#ai #ai-gateway #api-gateway #cncf #inference #kubernetes

Updated: 2026-09-07
Lists: 1 list mention
First commit: 2024-10-21
History: 7 history points
License: Apache-2.0
Issues: 282 open

2,003

stars

Forks: 363
Commits: 1,570 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Human-Agent-Society/reef

Continual learning infra for self-improving agents

AI dev

Stack

Python Next.js pytest React npm PEP 517 uv

GitHub topics

#agent-infrastructure #ai-agents #continual-learning #inference #llm #llm-training

Updated: 2026-09-15
Lists: 3 list mentions
First commit: 2026-08-31
History: 1 history point
License: Apache-2.0
Issues: 58 open

1,698

stars

Forks: 108
Commits: 286 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

xLLM-AI/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

AI dev

Stack

C++ Astro Vite CMake npm PEP 517

GitHub topics

#deepseek #glm #inference #inference-engine #large-language-models #llm-inference

Updated: 2026-10-17
Lists: 0 list mentions
First commit: 2025-08-13
History: 84 history points
License: Apache-2.0
Issues: 220 open

1,600

stars

Forks: 313
Commits: 1,909 commits
Star growth, last 7 days: +6 +0.4%
Commit velocity, last 7 days: +87 +4.8%

Website GitHub

utkuozdemir/nvidia_gpu_exporter

Nvidia GPU exporter for prometheus using nvidia-smi binary OR using NVML

AI dev

Stack

Go Go modules

GitHub topics

#ai #cryptocurrency #gaming #inference #llm #llm-training

Updated: 2026-10-15
Lists: 1 list mention
First commit: 2021-06-08
History: 1 history point
License: MIT
Issues: 7 open

1,568

stars

Forks: 156
Commits: 970 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

jd-opensource/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

AI dev

Stack

C++ CMake PEP 517 pip

GitHub topics

#deepseek #glm #inference #inference-engine #large-language-models #llm-inference

Updated: 2026-06-27
Lists: 1 list mention
First commit: 2025-08-13
History: 4 history points
License: Apache-2.0
Issues: 162 open

1,370

stars

Forks: 240
Commits: 1,115 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

AI dev

Stack

Cuda React Spring Boot Vite Bundler CMake Composer

GitHub topics

#gpt #inference #llama #llm #llm-serving #llmops

Updated: 2026-09-07
Lists: 1 list mention
First commit: 2023-12-27
History: 7 history points
License: Apache-2.0
Issues: 216 open

1,326

stars

Forks: 270
Commits: 4,752 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

kaito-project/aikit

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

AI dev

Stack

Go gRPC Go React Go modules npm Yarn

GitHub topics

#ai #buildkit #chatgpt #docker #fine-tuning #finetuning

Updated: 2026-09-25
Lists: 2 list mentions
First commit: 2023-09-20
History: 7 history points
License: MIT
Issues: 36 open

539

stars

Forks: 57
Commits: 776 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

defilantech/LLMKube

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

AI dev

Stack

Go Cobra gRPC Go Go modules

GitHub topics

#ai #apple-silicon #autoscaling #edge-computing #gguf #gpu

Updated: 2026-10-07
Lists: 1 list mention
First commit: 2025-11-17
History: 8 history points
License: Apache-2.0
Issues: 103 open

231

stars

Forks: 34
Commits: 1,169 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

pyadav/gateway

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

Stack

Go Cobra gRPC Go Next.js React Bun Go modules npm

GitHub topics

#ai #aistudio #api #gateway #generative-ai #grpc

Updated: 2024-04-08
Lists: 0 list mentions
First commit: 2024-01-18
History: 22 history points
License: Apache-2.0
Issues: 10 open

162

stars

Forks: 16
Commits: 141 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

missingstudio/gateway

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

Stack

Go Bun Go modules npm

GitHub topics

#ai #aistudio #api #gateway #generative-ai #grpc

Updated: 2024-04-08
Lists: 0 list mentions
First commit: 2024-01-18
History: 41 history points
License: Apache-2.0
Issues: 10 open

160

stars

Forks: 17
Commits: 141 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

CogitatorTech/infera

A DuckDB extension for in-database inference

AI dev

Stack

Rust pytest Cargo CMake PEP 517

GitHub topics

#duckdb #duckdb-extension #inference #machine-learning #onnx #rust

Updated: 2026-07-22
Lists: 1 list mention
First commit: 2025-09-26
History: 2 history points
License: Apache-2.0
Issues: 5 open

136

stars

Forks: 6
Commits: 40 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

benjilu/forestError

A Unified Framework for Random Forest Prediction Error Estimation

Stack

GitHub topics

#inference #intervals #machine-learning #machinelearning #prediction #r

Updated: 2021-08-10
Lists: 1 list mention
First commit: 2018-02-15
History: 1 history point
License: Unknown
Issues: 0 open

stars

Forks: 4
Commits: 111 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

cognisoc/zigllm

Learn how LLMs work by building one in Zig -- from tensors to text generation.

Stack

Zig pip

GitHub topics

#cognisoc #educational #from-scratch #inference #learn-by-doing #llm

Updated: 2026-07-02
Lists: 1 list mention
First commit: 2026-03-15
History: 6 history points
License: MIT
Issues: 1 open

stars

Forks: 0
Commits: 14 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing