Awesome Nlp

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

A high-throughput and memory-efficient inference and serving engine for LLMs

SGLang is a high-performance serving framework for large language models and multimodal models.

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Supercharge Your LLM Application Evaluations 🚀

Retrieval and Retrieval-augmented LLMs

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

State of the Art Natural Language Processing

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

Underthesea - AI Assistant

Data and tools for generating and inspecting OLMo pre-training data.

LongBench v2 and LongBench (ACL 25'&24')

PhoGPT: Generative Pre-training for Vietnamese (2023)

Deliver safe & effective language models

An NLP library for Uralic languages such as Finnish, Skolt Sami, Moksha and so on. Also supporting some non-Uralic languages such as Spanish, French, Arabic, Swedish, Norwegian, Russian and English. LLMs, FSTs and More!

Latxa: An Open Language Model and Evaluation Suite for Basque

Tiny QA Benchmark++ a micro-benchmark suite (52-item gold + on-demand multilingual synthetic packs), generator CLI, and CI-ready eval harness for ultra-fast LLM smoke-testing & regression-catching.