Awesome Nlp

LLM inference in C/C++

Unsupervised text tokenizer for Neural Network-based text generation.

Learning embeddings for classification, retrieval and ranking.

MITIE: library and tools for information extraction

Fast Neural Machine Translation in C++

Kiwi(지능형 한국어 형태소 분석기)

A Modern C++ Data Sciences Toolkit

Fast topic modeling platform

UDPipe: Trainable pipeline for tokenizing, tagging, lemmatizing and parsing Universal Treebanks and other CoNLL-U files

Colibri core is an NLP tool as well as a C++ and Python library for working with basic linguistic constructions such as n-grams and skipgrams (i.e patterns with one or more gaps, either of fixed or dynamic size) in a quick and memory-efficient way. At the core is the tool ``colibri-patternmodeller`` whi ch allows you to build, view, manipulate and query pattern models.

Frog is an integration of memory-based natural language processing (NLP) modules developed for Dutch. All NLP modules are based on Timbl, the Tilburg memory-based learning software package.

Unicode tokeniser. Ucto tokenizes text files: it separates words from punctuation, and splits sentences. It offers several other basic preprocessing steps such as changing case that you can all use to make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation. Ucto comes with tokenisation rules for several languages and can be easily extended to suit other languages. It has been incorporated for tokenizing Dutch text in Frog, our Dutch morpho-syntactic processor. http://ilk.uvt.nl/ucto --

InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.

NLP engine code

FoLiA library for C++