Awesome Nlp

OpenIE6 system

Pretraining, fine-tuning and evaluation scripts for IndicBERT-v2 and IndicXTREME

Pretraining transformer based Thai language models

State of the Art Language models and Classifier for Hindi language (spoken in Indian sub-continent)

Large Arabic Resources For Sentiment Analysis

A Go Lang package for dealing with Arabic text.

IndoLEM is a comprehensive Indonesian NLU benchmark, comprising three pillars NLP task: morpho-syntax, semantic, and discourse. Presented in COLING 2020.

FoLiA Linguistic Annotation Tool -- Flat is a web-based linguistic annotation environment based around the FoLiA format (http://proycon.github.io/folia), a rich XML-based format for linguistic annotation. Flat allows users to view annotated FoLiA documents and enrich these documents with new annotations, a wide variety of linguistic annotation types is supported through the FoLiA paradigm.

A collaborative project to collect datasets in SEA languages, SEA regions, or SEA cultures.

Unannotated Spanish 3 Billion Words Corpora

BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese (INTERSPEECH 2022)

An NLP library for Uralic languages such as Finnish, Skolt Sami, Moksha and so on. Also supporting some non-Uralic languages such as Spanish, French, Arabic, Swedish, Norwegian, Russian and English. LLMs, FSTs and More!

High performance tokenizers for natural language processing and other related tasks

Take a MALLET to disciplinary history

Indonesian Manually Tagged Corpus

cleanning up Persian text!

Frog is an integration of memory-based natural language processing (NLP) modules developed for Dutch. All NLP modules are based on Timbl, the Tilburg memory-based learning software package.

A Multilingual Latent Dirichlet Allocation (LDA) Pipeline with Stop Words Removal, n-gram features, and Inverse Stemming, in Python.

To analyze and remove gender bias in coreference resolution systems

A benchmark dataset for Indonesian text summarization.

An efficient and flexible token-based regular expression language and engine.

A keyphrase extractor for Persian

Collection of Urdu datasets for POS, NER, Sentiment, Summarization and NLP tasks.

Unicode tokeniser. Ucto tokenizes text files: it separates words from punctuation, and splits sentences. It offers several other basic preprocessing steps such as changing case that you can all use to make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation. Ucto comes with tokenisation rules for several languages and can be easily extended to suit other languages. It has been incorporated for tokenizing Dutch text in Frog, our Dutch morpho-syntactic processor. http://ilk.uvt.nl/ucto --

A Domain Specific Language (DSL) for building language patterns. These can be later compiled into spaCy patterns, pure regex, or any other format

No description.

Code related to the Dutch instance and user groups of the KALDI speech recognition toolkit

InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.

Saul : Declarative Learning-Based Programming

Named-Entity Recognition in Persian Language

Julia interface to word2vec

Code and models for "The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models". EACL 2021, WANLP.

PyTorch implementation of the end-to-end coreference resolution model with different higher-order inference methods.

A package for working with human languages

Spanish Billion Word Corpus and Embeddings

Parsing the Persian wikipedia's markup language into JSON and the most modern deep learning models built on it.

The English-Vietnamese Bilingual Corpus (EVBCorpus) is a collection of English and Vietnamese parallel translations and bitexts.

Python bindings to the dutch NLP tool Frog (pos tagger, lemmatiser, NER tagger, morphological analysis, shallow parser, dependency parser)

Indonesian conversion

An R package for exploring topic models of text

LABR: Large Scale Arabic Book Reviews Dataset

spanlp: nlp applied for spanish vulgarity. A fast, robust Python library to check for profanity or offensive language in Spanish strings. It contains all the rude words of Spanish-speaking countries.

Contains demo codes for the course Applied Natural Language Processing

A collection of links to Ukrainian language tools

Thai Word Segmentation and Part-of-Speech Tagging with Deep Learning

청와대 국민청원 데이터

R package wrapping Mallet

No description.

Latxa: An Open Language Model and Evaluation Suite for Basque

ViText2SQL: A dataset for Vietnamese Text-to-SQL semantic parsing (EMNLP-2020 Findings)