Awesome Observability

SigNoz is an open-source, OpenTelemetry-native observability platform for your team and their AI agents. Get logs, metrics, and traces in one tool with features like APM, distributed tracing, log management, infra monitoring, etc. Combined with SigNoz MCP and a native AI teammate (in SigNoz Cloud) it helps you build more resilient apps.

Write scalable load tests in plain Python 🚗💨

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash

AI Observability & Evaluation

Easy & Flexible Alerting With ElasticSearch

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Agenta is a workspace where you and your team build agents and automations.

AI observability platform for production LLM and agent systems.

OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.

Load and performance benchmark tool

ML powered analytics engine for outlier detection and root cause analysis.

Track AI API quotas across Synthetic, Z.ai, Anthropic (Claude Code), Codex, GitHub Copilot & Antigravity in real time. Lightweight background daemon (<50MB RAM), SQLite storage, Material Design 3 dashboard. Zero telemetry.

Log Anomaly Detection - Machine learning to detect abnormal events logs

Streaming anomaly detection in Python: 16 online detectors (xStream, LODA, RS-Hash, Half-Space Trees, RRCF…) plus calibration and prequential evaluation for real-time data.

OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team.

Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk

Unified observability for logs, metrics and traces — low overhead instrumentation (Rust, Unreal Engine 5), OTLP ingestion, SQL analytics on DataFusion over Parquet, presentation through notebooks, Grafana and Python. Self-hosted.

Free, read-only GPU waste scanner for Kubernetes inference clusters, by Paralleliq. Finds idle GPUs, oversized GPU tiers and wasted capacity in vLLM deployments.

Python SDK for accurate and verifiable agent tool use. Agents verify that answers came from the right source and were not changed. Downstream agents detect 100% of errors and retry to achieve a 50% jump in answer accuracy.

Open, reproducible benchmarks for crypto infrastructure: aggregators, bridges, blockchains, perps. Live data, public methodology.

Weckr SDKs and Claude Skills: TypeScript, Python, MCP server, and an installable Claude Code plugin. AI cost and margin intelligence for SaaS, per user, per feature, per LLM call.

Real-time Prometheus exporter for Duplicacy backups with live speed, progress, and completion metrics

Local-first observability dashboard for AI agents. MCP-native. Look at every span your agents emit.

Local replay debugger for Browser Use failures with screenshots, model I/O, failed-step timelines, and public-safe HTML exports.

Chat-driven Prometheus visualization with one-click Grafana export. Self-hosted, single image, bring your own LLM.

Deterministic pytest suite generator from Kibana API logs

Send unhandled Python exceptions straight to a Telegram chat. No backend, no account, no database - just your own bot token.