Awesome Llm

The platform for LLM evaluations and AI agent testing

A quick guide (especially) for trending instruction finetuning datasets

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

The hub for EleutherAI's work on interpretability and learning dynamics

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

Minimalistic large language model 3D-parallelism training

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

A simple, performant, and scalable Jax LLM!

Seamlessly integrate LLMs as Python functions

OpenAGI: When LLM Meets Domain Experts

Mesh TensorFlow: Model Parallelism Made Easier

MemFree - Hybrid AI Search Engine & AI Page Generator

Build production-ready LLM applications and advanced agents using Python, LangChain, and LangGraph. This is the companion repository for the book on generative AI with LangChain.

A security scanner for your LLM agentic workflows

Make your LLM agent and chat with it simple and fast!