Search awesome repositories

Mastra is the modern TypeScript framework for AI-powered applications and agents.

AI Observability & Evaluation

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Framework for evaluating and improving agents

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.

AI observability platform for production LLM and agent systems.

Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

AI system design guide for engineers building production AI systems and evals.

🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL

Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs

Diff your AI agent's behavior between two runs. See exactly which tool calls, args, costs and outputs changed when you swap models or edit prompts.

Local-first observability dashboard for AI agents. MCP-native. Look at every span your agents emit.

Local-first DevTools for AI calls. See every LLM call your app makes: prompt, cost, tokens, latency, tool calls.