Sign in
← Back to search
github Not archived

proycon/python-ucto

This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is regular-expression based, extensible, and advanced tokeniser written in C++ (http://ilk.uvt.nl/ucto).

2 awesome lists

Quick read

Stars
31
Forks
5
Open issues
5
Commits
153

Activity and growth

Latest capture 2026-11-09 09:52

Stars · last 7 days
0 0.0%
Commits · last 7 days
0 0.0%
Stars since tracking
0
Stored snapshots
1

Compare the starting count and capture dates, not just the percentage. How to read GitHub star growth. Missing history is not zero growth.

Classification

Topics

#computational-linguistics #folia #nlp #nlp-library #python #text-processing #tokenizer

Metadata

Language
Cython
Default branch
master
Created
2014-05-21
First commit
2014-05-21
Last pushed
2026-08-20
GitHub updated
2026-10-29
Last synced
2026-11-09 09:52
Stack scanned
2026-11-09 09:52
Archived
No

Archive status reflects the last catalog sync, not a maintenance assessment. Check recent releases, issue responses, and fit for your project with the repository shortlisting checklist.

Sign in to like proycon/python-ucto
Compare this repository README Open website View on GitHub

AI development signals

0 paths

Agent instructions and tool configuration found in this repository.

No config files detected.

Growth history

Tracked growth

1 observed capture since 2026-11-09. Observed captures are shown by default.

Stars from first capture 0

Chart data

Observed captures only

Time horizon

All tracked data

Custom date range

Stars history

Observed snapshots

Commits history

Observed snapshots

Similar repositories

Nearest indexed repositories by embedding similarity.

LanguageMachines/ucto

Unicode tokeniser. Ucto tokenizes text files: it separates words from punctuation, and splits sentences. It offers several other basic preprocessing steps such as changing case that you can all use to make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation. Ucto comes with tokenisation rules for several languages and can be easily extended to suit other languages. It has been incorporated for tokenizing Dutch text in Frog, our Dutch morpho-syntactic processor. http://ilk.uvt.nl/ucto --

72 stars
C++ 2 awesome lists

proycon/python-frog

Python bindings to the dutch NLP tool Frog (pos tagger, lemmatiser, NER tagger, morphological analysis, shallow parser, dependency parser)

49 stars
Cython 2 awesome lists

JuliaText/WordTokenizers.jl

High performance tokenizers for natural language processing and other related tasks

100 stars
Julia 2 awesome lists

jdkato/prose

:book: A Golang library for text processing, including tokenization, part-of-speech tagging, and named-entity extraction.

3,092 stars
Go 1 awesome list

proycon/foliapy

An extensive Python library for dealing with FoLiA (Format for Linguistic Annotation) documents, a rich XML-based format for linguistic annotation finding application in Natural Language Processing (NLP). This library was formerly part of PyNLPl.

17 stars
Python 1 awesome list

huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

11,229 stars
Rust 4 awesome lists