Awesome Harness Engineering

#1 Persistent memory for AI coding agents based on real-world benchmarks

The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents

Symphony turns project work into isolated, autonomous implementation runs, allowing teams to manage work instead of supervising coding agents.

SpaceXAI's coding agent harness and TUI. Fullscreen, mouse interactive, extensible.

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Agent2Agent (A2A) is an open protocol enabling communication and interoperability between opaque agentic applications.

Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Scale computer-use 2.0 with open-source drivers, cross-OS fleets, and benchmarks for training, evaluation, and data generation.

Context window optimization for AI coding agents. Sandboxes tool output (98% reduction), persists session memory, and enforces routing across 17 platforms via MCP + hooks.

An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

How Python does AI. Agents, realtime voice, image generation, embeddings. Every model, every interface, typed end to end.

SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]

The LLM Evaluation Framework

SkillOpt is a text-space optimizer that trains reusable natural-language skills for frozen LLM agents through trajectory-driven edits, validation-gated updates, and deployable best_skill.md artifacts.

A collection of projects designed to help developers quickly get started with building deployable applications using the Claude API

Browser Harness | Self-healing harness that enables LLMs to complete any task.

OpenWiki is a CLI that writes and maintains agent documentation for your codebase.

AG-UI: the Agent-User Interaction Protocol. Bring Agents into Frontend Applications.

"DeepCode: Open Agentic Coding (Agent Harness & Loop Engineering & Multi-Agent Orchestration)"

Structured Outputs

"OpenHarness: Open Agent Harness with a Built-in Personal Agent--Ohmo!"

Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.

Multiplayer agent harness for work.

Fully autonomous & self-evolving research from idea to paper. Chat an Idea. Get a Paper. 🦞

The best agent harness.

Open-source, secure environment with real-world tools for enterprise-grade agents.

IronClaw is an Agent OS focused on privacy, security and extensibility

Instant, Concurrent, Secure & Lightweight Sandbox for AI Agents.

Build your own AI SRE agents. The open source toolkit for the AI era.

AI Observability & Evaluation

Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost.

Free, local tool to track AI coding token usage and cost across 37 tools and agents (Claude Code, Cursor, Codex, Gemini and more), by model, project, and task. npx codeburn

Multi-Agent Harness for Production AI

Visual testing tool for MCP servers

Secure, Fast, and Extensible Sandbox runtime for AI agents.

Omnigent is an open-source AI agent framework and meta-harness: orchestrate Claude Code, Codex, Cursor, Pi, and custom agents — swap harnesses without rewriting, enforce policies and sandboxing, and collaborate in real time from any device.

A meta-skill that designs domain-specific agent teams, defines specialized agents, and generates the skills they use.

Build an agent harness and control it end-to-end. Open-source SDK for production AI agents in Python & TypeScript - any model, any cloud.

Build effective agents using Model Context Protocol and simple workflow patterns

OpenShell is the safe, private runtime for autonomous AI agents.

The sandbox agent framework.

Deepsec is a security harness for finding vulnerabilities in your codebase powered by coding agents

Orchestrate sandboxed coding agents in TypeScript with sandcastle.run()

Kimi Code CLI — The Starting Point for Next-Gen Agents

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

Persistent memory system for AI coding agents. Agent-agnostic Go binary with SQLite + FTS5, MCP server, HTTP API, CLI, and TUI.

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

Self-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it.

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.