Search awesome repositories

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.

Open Source LLM Gateway 🦚

Connect Your Agents And Harnesses With Any Provider 🦚

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.

Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces.

AI observability platform for production LLM and agent systems.

Agent Skills as a Memory Layer

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

DataBuff is an AI-native APM built on Opentelemetry，with multi-agent troubleshooting out of the box.

See your agent think. Zero-config observability & governance for 30 AI agent runtimes: Claude Code, OpenAI Codex, Hermes, OpenClaw & 26 more. Live token costs, sessions, tool calls, crons.

React components for visualizing traces from AI agents

OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.

Model Context Protocol (MCP) server for Opik, the open-source LLM observability and evaluation platform, built by Comet. Read traces, log scores, and manage prompts from Claude Code, Cursor, or VS Code.

Local-first MCP tools and dashboard for investigating Codex token usage, credits, costs, caching, and thread patterns.

the agent experience layer · observability + memory for AI coding agents (Claude Code + Codex) · local-first, typed, yours

Agent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude, Codex, DeepAgents and more — deployed on the provider side, no SDK changes required.

Open-source observability for your LLM application.

Open-source, local observability TUI for Claude Code and Codex — context, cost, tokens, agents, tools, compactions, and code changes.

Local-first observability dashboard for AI agents. MCP-native. Look at every span your agents emit.

Local replay debugger for Browser Use failures with screenshots, model I/O, failed-step timelines, and public-safe HTML exports.

Grafana-native LLMOps dashboards for FinOps, agents, quality and AI governance, generated and visually verified.