Awesome Harness Engineering

OpenShell is the safe, private runtime for autonomous AI agents.

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and reliability engineering for autonomous AI agents. Covers 10/10 OWASP Agentic Top 10.

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Fast and Accurate Code Search for Agents. Uses 99% fewer tokens than grep+read

The first "code-first" agent framework for seamlessly planning and executing data analytics tasks.

Build distributed, production-grade, long-running agents.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Nexent is a zero-code platform for auto-generating production-grade AI agents using Harness Engineering principles — unified tools, skills, memory, and orchestration with built-in constraints, feedback loops, and control planes.

Zep | Examples, Integrations, & More

AI observability platform for production LLM and agent systems.

agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes and reinforcement learning (RL).

The World's First Virtual Terminal for AI Agents

Amazon Bedrock Agentcore accelerates AI agents into production with the scale, reliability, and security, critical to real-world deployment.

Squad: AI agent teams for any project

Inspect: A framework for large language model evaluations

Official, AWS-supported MCP servers, skills, and plugins to help AI agents build on AWS

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

Reference code for the Meta-Harness paper.

Build, Evaluate, and Deploy GUI Agents — online RL training, standardized benchmarks, and real-device deployment in one framework.

a recursive self-improving harness designed to help your agents (and future iterations of those agents) succeed on any task

The open‑source AI Agents Governance & Orchestration framework: write the rules declaratively, Bernstein enforces them and produces the verifiable, replayable record. Free, Apache-2.0. https://bernstein.run

MCP server that gets Claude to 97.9% (188/192) on a real coding benchmark at -80% active tokens and -83% wall time, vs 78.3% plain. Structural code navigation + persistent memory engine. Works with every MCP client.

Minimal and readable coding agent harness implementation in Python to explain the core components of coding agents.

Weave is a toolkit for developing AI-powered applications, built by Weights & Biases.

Official AHE code — Agentic Harness Engineering: observability-driven automatic evolution of coding-agent harnesses (concurrent w/ meta-harness). NexAU-AHE reaches 84.7% ± 2.1 pass@1 on Terminal-Bench 2 (GPT-5.5). Lifts GPT-5.4 69.7→77.0% over 10 iters, beats Codex/ACE/Training-Free GRPO; frozen harness transfers to SWE-bench-Verified.

A production-ready runtime framework for agent apps with secure tool sandboxing, Agent-as-a-Service APIs, scalable deployment, full-stack observability, and broad framework compatibility.

Claw-Eval is an evaluation harness for evaluating LLM as agents. All tasks verified by humans.

Open HTTP protocol for human decisions in AI agent workflows. Service-hosted reviews, structured approvals, selections and forms, with optional messenger actions.