Awesome Nlp

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

LLM inference in C/C++

Robust Speech Recognition via Large-Scale Weak Supervision

No description.

A high-throughput and memory-efficient inference and serving engine for LLMs

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

SGLang is a high-performance serving framework for large language models and multimodal models.

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

结巴中文分词

💫 Industrial-strength Natural Language Processing (NLP) in Python

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Label Studio is a multi-type data labeling and annotation tool with standardized output format

Qwen3 is the large language model series developed by Qwen team, Alibaba Cloud.

Library for fast text representation and classification.

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

A framework for few-shot evaluation of language models.

A very simple framework for state-of-the-art Natural Language Processing (NLP)

ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型

Retrieval and Retrieval-augmented LLMs

Unsupervised text tokenizer for Neural Network-based text generation.

Foundational Models for State-of-the-Art Speech and Text Translation

An open-source NLP research library, built on PyTorch.

Large Language Model Text Generation Inference

Open source annotation tool for machine learning practitioners.

YSDA course in Natural Language Processing

Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages

A series of large language models trained from scratch by developers @01-ai

Python library for processing Chinese text

An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more

Language Technology Platform

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

The RedPajama-Data repository contains code for preparing large datasets for training large language models.

Aligning pretrained language models with instruction data generated by themselves.

Reading Wikipedia to Answer Open-Domain Questions

A series of large language models developed by Baichuan Intelligent Technology

Snips Python library to extract meaning from text

TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Top2Vec learns jointly embedded topic, document and word vectors.

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

MITIE: library and tools for information extraction

A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Efficient few-shot learning with Sentence Transformers

NLP made easy

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Machine Learning University: Accelerated Natural Language Processing Class

Kashgari is a production-level NLP Transfer learning framework built on top of tf.keras for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.