Awesome Nlp

DaNLP is a repository for Natural Language Processing resources for the Danish Language.

Morfessor is a tool for unsupervised and semi-supervised morphological segmentation

StereoSet: Measuring stereotypical bias in pretrained language models

Comprehensive NLP Evaluation System

A massively multilingual modern encoder language model

Neural network models for joint POS tagging and dependency parsing (CoNLL 2017-2018)

Thai Word-Segmentation with LSTM in Tensorflow

คลังศัพท์ภาษาไทย

[EMNLP-2024] ⚓️ Sailor: Open Language Models for South-East Asia

OpenIE6 system

Pretraining, fine-tuning and evaluation scripts for IndicBERT-v2 and IndicXTREME

Large Arabic Resources For Sentiment Analysis

IndoLEM is a comprehensive Indonesian NLU benchmark, comprising three pillars NLP task: morpho-syntax, semantic, and discourse. Presented in COLING 2020.

A collaborative project to collect datasets in SEA languages, SEA regions, or SEA cultures.

Unannotated Spanish 3 Billion Words Corpora

An NLP library for Uralic languages such as Finnish, Skolt Sami, Moksha and so on. Also supporting some non-Uralic languages such as Spanish, French, Arabic, Swedish, Norwegian, Russian and English. LLMs, FSTs and More!

A Multilingual Latent Dirichlet Allocation (LDA) Pipeline with Stop Words Removal, n-gram features, and Inverse Stemming, in Python.

A benchmark dataset for Indonesian text summarization.

A keyphrase extractor for Persian

A Domain Specific Language (DSL) for building language patterns. These can be later compiled into spaCy patterns, pure regex, or any other format

No description.

Code and models for "The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models". EACL 2021, WANLP.

PyTorch implementation of the end-to-end coreference resolution model with different higher-order inference methods.

Spanish Billion Word Corpus and Embeddings

LABR: Large Scale Arabic Book Reviews Dataset

spanlp: nlp applied for spanish vulgarity. A fast, robust Python library to check for profanity or offensive language in Spanish strings. It contains all the rude words of Spanish-speaking countries.

Thai Word Segmentation and Part-of-Speech Tagging with Deep Learning

청와대 국민청원 데이터

A high-accuracy, multilingual rule-based Sentence Boundary Detector (SBD) with a drop-in adapter for pysbd, delivering faster and more accurate sentence segmentation.

Platform for Evaluating and Reviewing of Multilingual Tasks

DRF-based API server for Shoonya platform

Natural language toolkit for Indonesian Language (Bahasa)

Tiny QA Benchmark++ a micro-benchmark suite (52-item gold + on-demand multilingual synthetic packs), generator CLI, and CI-ready eval harness for ultra-fast LLM smoke-testing & regression-catching.

An extensive Python library for dealing with FoLiA (Format for Linguistic Annotation) documents, a rich XML-based format for linguistic annotation finding application in Natural Language Processing (NLP). This library was formerly part of PyNLPl.

Aspect based sentiment analysis for Hindi

sentiment analysis for spacy pipeline in python

Bilingual RAG evaluation benchmark for culturally grounded English/Uzbek retrieval

No description.