Awesome Opensource Ai

Nano vLLM

Unified framework for building enterprise RAG pipelines with small, specialized models

Unity MCP acts as a bridge between AI assistants and your Unity Editor. Give your LLM tools to manage assets, control scenes, edit scripts, and automate tasks within Unity.

Desktop Companion for Hermes Agent

Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D avatar running locally across platforms

Graph-Native Infrastructure for Context and Accountable AI Systems

Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

LangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and vector stores, and makes implementing tool calling (including MCP support), agents and RAG easy. It integrates seamlessly with enterprise Java frameworks like Quarkus and Spring Boot.

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

HexStrike AI MCP Agents is an advanced MCP server that lets AI agents (Claude, GPT, Copilot, etc.) autonomously run 150+ cybersecurity tools for automated pentesting, vulnerability discovery, bug bounty automation, and security research. Seamlessly bridge LLMs with real-world offensive security capabilities.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Go ahead and axolotl questions

Retrieval and Retrieval-augmented LLMs

Trae Agent is an LLM-based agent for general purpose software engineering tasks.

Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/

Low-code framework for building custom LLMs, neural networks, and other AI models

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

Connect your users to their apps

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

The open source codebase powering HuggingChat

Official inference library for Mistral models

AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework

An Open-Source Asynchronous Coding Agent

All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.

Build, Manage and Deploy AI/ML Systems

Cybersecurity AI (CAI), the framework for AI Security

High-speed Large Language Model Serving for Local Deployment

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

⚙️🦀 Build modular and scalable LLM Applications in Rust

Build an agent harness and control it end-to-end. Open-source SDK for production AI agents in Python & TypeScript - any model, any cloud.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Build effective agents using Model Context Protocol and simple workflow patterns

Accessible large language models via k-bit quantization for PyTorch.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Fast, flexible LLM inference

🔥 Open Source Browser API for AI Agents & Apps. Steel Browser is a batteries-included browser sandbox that lets you automate the web without worrying about infrastructure.

Adding guardrails to large language models.

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

Dynamic, resilient AI orchestration. Coordinate data, models, and compute as you build AI workflows.

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Tools for merging pretrained large language models.

GLM-5: From Vibe Coding to Agentic Engineering

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷