Awesome Chatgpt

A high-throughput and memory-efficient inference and serving engine for LLMs

CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.

The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.

Hundreds of models & providers. One command to find what runs on your hardware.

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

SGLang is a high-performance serving framework for large language models and multimodal models.

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.

A powerful GUI app and Toolkit for Claude Code - Create custom agents, manage interactive Claude Code sessions, run secure background agents, and more.

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

Open source observability platform for logs, metrics, traces, RUM, Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.

The open-source AI Operating System

Open-source Agent Operating System

🌐 The open-source Agentic browser; alternative to ChatGPT Atlas, Perplexity Comet, Dia.

Capture first. Organize later. A local-first Markdown app that turns scattered records into clear notes with AI.

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

Incremental engine for long horizon agents 🌟 Star if you like it!

Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harnesses without rewriting, enforce policies and sandboxing, and collaborate in real time from any device.

Astrid is a portable, capability-secure operating system for composable software.

World's first open-source enterprise world model.

The programming language for agents

⚙️🦀 Build modular and scalable LLM Applications in Rust

Fast, flexible LLM inference

AI enabled pair programmer for Claude, GPT, O Series, Grok, Deepseek, Gemini and 300+ models

An AI-powered next-generation open source real-time observability system.

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

🔥 PlainApp is an open-source app that lets you securely manage your phone from a web browser. Access files, media, contacts, SMS, calls, and more through a simple, easy-to-use interface on your desktop.

Postgres with GPUs for ML/AI apps.

A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

A blazing fast inference solution for text embeddings models

Enterprise AI Platform with guardrails, MCP registry, gateway & orchestrator

eBPF Observability - Distributed Tracing and Profiling

LeanCTX — Context Gateway for AI Systems. Control what your AI can see. Open-source Engine for context selection, supported controls, and evidence.

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Hold a key, speak, release — AI-polished text appears at your cursor in any app. Open-source voice input for macOS & Windows. (按住快捷键说话，松开即得润色后的文字)

Manage multiple Claude Code, OpenCode agents from either TUI or Web for easy access on mobile. Also supports Mistral Vibe, Codex CLI, Gemini CLI, Pi.dev, Copilot CLI, Factory Droid Coding.

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

Manage ChatGPT, Codex CLI, and Claude Code accounts and track quota in one place. Use multi-model smart routing with priority and automatic failover. Install AI tools and deploy local LLMs in one click.

Open-source inference server and production cluster for all the models your agent needs.

Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.

Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention

A secure persistent personal agent server in Rust. One binary, sandboxed execution, multi-provider LLMs, voice, memory, Telegram, WhatsApp, Discord, Teams, and MCP tools. Secure by design, runs on your hardware.

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

All-in-one platform for search, recommendations, RAG, and analytics offered via API