Awesome Nlp

NLP, before and after spaCy

Pre-trained word vectors of 30+ languages

Basic Utilities for PyTorch Natural Language Processing (NLP)

Underthesea - AI Assistant

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

This repository includes the official implementation of OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs.

Measuring Massive Multitask Language Understanding | ICLR 2021

Data and tools for generating and inspecting OLMo pre-training data.

Pre-trained ELMo Representations for Many Languages

Persian NLP Toolkit

Korean BERT pre-trained cased (KoBERT)

Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons

LongBench v2 and LongBench (ACL 25'&24')

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

Morphological analyzer / inflection engine for Russian and Ukrainian languages.

Thai natural language processing in Python

Data repository for pretrained NLP models and NLP corpora.

Expanding natural instructions

The Classical Language Toolkit

Portuguese pre-trained BERT models

Natural Language Toolkit for Indic Languages aims to provide out of the box support for various NLP tasks that an application developer might need

Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

PhoGPT: Generative Pre-training for Vietnamese (2023)

A Neural Framework for MT Evaluation

Facebook Low Resource (FLoRes) MT Benchmark

Pre-trained Transformers for Arabic Language Understanding and Generation (Arabic BERT, Arabic GPT2, Arabic ELECTRA)

Dataset and codes for ACL 2019 DocRED: A Large-Scale Document-Level Relation Extraction Dataset.

Resources and tools for Indian language Natural Language Processing

Hierarchical unsupervised and semi-supervised topic models for sparse count data with CorEx

SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models

Accurately generate all possible forms of an English word e.g "election" --> "elect", "electoral", "electorate" etc.

Naver sentiment movie corpus

A suite of Arabic natural language processing tools developed by the CAMeL Lab at New York University Abu Dhabi.

REBEL is a seq2seq model that simplifies Relation Extraction (EMNLP 2021).

Deliver safe & effective language models

Library for translating between 200 languages. Built on 🤗 transformers.

Translation models for 22 scheduled languages of India

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

KSS: Korean String processing Suite

A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"

potato: the portable annotation tool

No description.

Indonesian stemmer. Python port of PHP Sastrawi project.

A Python library for Interpretable Machine Learning in Text Classification using the SS3 model, with easy-to-use visualization tools for Explainable AI :octocat:

An NLP library for the Urdu language. It comes with a lot of battery included features to help you process Urdu data in the easiest way possible.

No description.

A Language Processing Toolkit for Persian

The tool to make NLP datasets ready to use

FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes

Repository for research in the field of Responsible NLP at Meta.