Awesome Nlp

Data repository for pretrained NLP models and NLP corpora.

Expanding natural instructions

Self-contained Japanese Morphological Analyzer written in pure Go

Natural Language Interface for apps and devices

TruthfulQA: Measuring How Models Imitate Human Falsehoods

The Classical Language Toolkit

Portuguese pre-trained BERT models

Fast vectorization, topic modeling, distances and GloVe word embeddings in R.

Natural Language Toolkit for Indic Languages aims to provide out of the box support for various NLP tasks that an application developer might need

The most accurate natural language detection library for Java and the JVM, suitable for long and short text alike

PhoBERT: Pre-trained language models for Vietnamese (EMNLP-2020 Findings)

Kiwi(지능형 한국어 형태소 분석기)

Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

PhoGPT: Generative Pre-training for Vietnamese (2023)

A Neural Framework for MT Evaluation

Facebook Low Resource (FLoRes) MT Benchmark

Natural Language Processing in Clojure (opennlp)

Pre-trained Transformers for Arabic Language Understanding and Generation (Arabic BERT, Arabic GPT2, Arabic ELECTRA)

Revisiting Pre-trained Models for Chinese Natural Language Processing (MacBERT)

A Modern C++ Data Sciences Toolkit

A Vietnamese natural language processing toolkit (NAACL 2018)

Fast topic modeling platform

The first-ever vast natural language processing benchmark for Indonesian Language. We provide multiple downstream tasks, pre-trained IndoBERT models, and a starter code! (AACL-IJCNLP 2020)

Dataset and codes for ACL 2019 DocRED: A Large-Scale Document-Level Relation Extraction Dataset.

Resources and tools for Indian language Natural Language Processing

Hierarchical unsupervised and semi-supervised topic models for sparse count data with CorEx

SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models

Accurately generate all possible forms of an English word e.g "election" --> "elect", "electoral", "electorate" etc.

Naver sentiment movie corpus

NLP in Python with Deep Learning

📖 Korean NLU Benchmark

A suite of Arabic natural language processing tools developed by the CAMeL Lab at New York University Abu Dhabi.

REBEL is a seq2seq model that simplifies Relation Extraction (EMNLP 2021).

The hands-on NLTK tutorial for NLP in Python

Natural Language Processing Pipeline - Sentence Splitting, Tokenization, Lemmatization, Part-of-speech Tagging and Dependency Parsing

Deliver safe & effective language models

Web-Scale Open Information Extraction

A web-based annotation tool for natural language processing (NLP)

BETO - Spanish version of the BERT model

Library for translating between 200 languages. Built on 🤗 transformers.

Translation models for 22 scheduled languages of India

Polyglot: Large Language Models of Well-balanced Competence in Multi-languages

CogComp's Natural Language Processing Libraries and Demos: Modules include lemmatizer, ner, pos, prep-srl, quantifier, question type, relation-extraction, similarity, temporal normalizer, tokenizer, transliteration, verb-sense, and more.

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

**Archived** Epic is a high performance statistical parser written in Scala, along with a framework for building complex structured prediction models.

KSS: Korean String processing Suite

Evaluating Cross-lingual Sentence Representations

Fast and production-ready question answering in Node.js

A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"

potato: the portable annotation tool