Awesome Opensource Ai

OpenAgents - The collaboration OS for AI agents

Evaluation and Tracking for LLM Experiments and AI Agents

DeepResearchAgent is a hierarchical multi-agent system designed not only for deep research tasks but also for general-purpose task solving. The framework leverages a top-level planning agent to coordinate multiple specialized lower-level agents, enabling automated task decomposition and efficient execution across diverse and complex domains.

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Build production-ready AI agents in both Python and Typescript.

RAG Web UI is an intelligent dialogue system based on RAG (Retrieval-Augmented Generation) technology.

A framework for prompt tuning using Intent-based Prompt Calibration

Inspect: A framework for large language model evaluations

OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

An AI-powered agentic red team framework that automates offensive security operations, from reconnaissance to exploitation to post-exploitation, with zero human intervention.

Developer Asset Hub for NVIDIA Nemotron — A one-stop resource for training recipes, usage cookbooks, datasets, and full end-to-end reference examples to build with Nemotron models

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering

Synthetic data curation for post-training and structured data extraction

A fast and lightweight framework for creating decentralized agents with ease.

Reference implementations of MLPerf® inference benchmarks

Readymade evaluators for your LLM apps