Awesome Agent Harness

Evaluation and Tracking for LLM Experiments and AI Agents

Devon: An open-source pair programmer

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Amazon Bedrock Agentcore accelerates AI agents into production with the scale, reliability, and security, critical to real-world deployment.

Agent harness to make your slop code well-engineered and beautiful.

Security scanner for AI agents, MCP servers and agent skills.

A model-driven harness for AI coding agents: semantic modeling, app generation, progressive API Assist and evidence-based verification.

The NVIDIA NeMo Agent toolkit is an open-source library for efficiently connecting and optimizing teams of AI agents.

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

One brain, many harnesses. Portable .agent/ folder (memory + skills + protocols) that plugs into Claude Code, Cursor, Windsurf, OpenCode, OpenClaw, Hermes, or DIY Python — and keeps its knowledge when you switch.

τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

Continual learning infra for self-improving agents

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"

Reference code for the Meta-Harness paper.

E2B Desktop Sandbox for LLMs. E2B Sandbox with desktop graphical environment that you can connect to any LLM for secure computer use.

LLM powered fuzzing via OSS-Fuzz.

OpenTelemetry Instrumentation for AI Observability

Evaluate and improve models and agents using environments

Minimal and readable coding agent harness implementation in Python to explain the core components of coding agents.

The Cloud Sandbox Built for AI Agents

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

Open-source benchmark for browser AI agents on daily tasks.

Tensorlake is a serverless runtime for sandboxes and deploying background agentic applications

Batteries for your Pydantic AI agent.

Official AHE code — Agentic Harness Engineering: observability-driven automatic evolution of coding-agent harnesses (concurrent w/ meta-harness). NexAU-AHE reaches 84.7% ± 2.1 pass@1 on Terminal-Bench 2 (GPT-5.5). Lifts GPT-5.4 69.7→77.0% over 10 iters, beats Codex/ACE/Training-Free GRPO; frozen harness transfers to SWE-bench-Verified.

CUGA is an open-source generalist agent harness for the enterprise, supporting complex task execution on web and APIs, OpenAPI/MCP integrations, composable architecture, reasoning modes, and policy-aware features.

No description.

A production-ready runtime framework for agent apps with secure tool sandboxing, Agent-as-a-Service APIs, scalable deployment, full-stack observability, and broad framework compatibility.

An agent benchmark with tasks in a simulated software company.

Claw-Eval is an evaluation harness for evaluating LLM as agents. All tasks verified by humans.

CheetahClaws: A Fast and Easy-to-Use Agent Harness Infrastructure for Long-Horizon, Multi-Model, and Tool-Using AI Systems

Collection of evals for Inspect AI

Measuring and evolving with the frontier of agent work

Sandboxed code execution for AI agents, locally or on the cloud. Massively parallel, easy to extend. Powering SWE-agent and more.

Bring your own agent and build a self-improving agentic system. Automatically mine failures, optimize the agent harness, and gate against regressions.

SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?

An in-the-wild benchmark for AI agents in the production harness.

Deterministic safety solutions for probabilistic AI agents

A generative AI-powered framework for testing virtual agents.

The production-ready agent harness framework for Python

WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?

Open Python agent harness for production AI apps: tools, MCP, memory, workspace, telemetry, subagents, background tasks, and OmniServe APIs.

Secure local dev environment for collaboration with AI coding agents

HexAgent – An Agent harness that gives any LLM a computer to complete tasks the way humans do

Evaluation harness for OpenHands V1.

No description.

Benchmark for comparing agent harnesses on everyday online tasks — fixes the base model, varies the harness. Sister project of ClawBench, same scoring pipeline.

A verified version of the WebArena Benchmark