Awesome Nlp

Reading Wikipedia to Answer Open-Domain Questions

State of the Art Natural Language Processing

A series of large language models developed by Baichuan Intelligent Technology

Snips Python library to extract meaning from text

Learning embeddings for classification, retrieval and ranking.

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

A Code-First Introduction to NLP course

TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Vietnamese TTS with instant voice cloning • On-device • Real-time CPU inference • 48kHz audio quality • Chuyển văn bản thành giọng nói tiếng Việt • Text to speech tiếng Việt • TTS tiếng Việt

Twitter Text Libraries. This code is used at Twitter to tokenize and parse text to meet the expectations for what can be used on the platform.

Top2Vec learns jointly embedded topic, document and word vectors.

:book: A Golang library for text processing, including tokenization, part-of-speech tagging, and named-entity extraction.

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

MITIE: library and tools for information extraction

A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Efficient few-shot learning with Sentence Transformers

中文自然语言处理工具包 Toolkit for Chinese natural language processing

"结巴"中文分词的Golang版本

NLP made easy

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Unified Schema-Based Information Extraction

Machine Learning University: Accelerated Natural Language Processing Class

natural language processor powered by plugins part of the @unifiedjs collective

Doing simple retrieval from LLM models at various context lengths to measure accuracy

Kashgari is a production-level NLP Transfer learning framework built on top of tf.keras for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.

Beautiful visualizations of how language differs among document types.

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

NLP, before and after spaCy

Pre-trained word vectors of 30+ languages

Basic Utilities for PyTorch Natural Language Processing (NLP)

Code and data accompanying Natural Language Processing with PyTorch published by O'Reilly Media https://amzn.to/3JUgR2L

BERT score for text generation

Underthesea - AI Assistant

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

This repository includes the official implementation of OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs.

Measuring Massive Multitask Language Understanding | ICLR 2021

Data and tools for generating and inspecting OLMo pre-training data.

Fast Neural Machine Translation in C++

Pre-trained ELMo Representations for Many Languages

Persian NLP Toolkit

Korean BERT pre-trained cased (KoBERT)

A collection of links to Ruby Natural Language Processing (NLP) libraries, tools and software

Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons

LongBench v2 and LongBench (ACL 25'&24')

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

Text mining using tidy tools :sparkles::page_facing_up::sparkles:

Morphological analyzer / inflection engine for Russian and Ukrainian languages.

Thai natural language processing in Python

Natural language detection library for Rust. Try demo online: https://whatlang.org/