huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
:book: A Golang library for text processing, including tokenization, part-of-speech tagging, and named-entity extraction.
Appears on
Quick read
Latest capture 2026-11-09 09:47
Compare the starting count and capture dates, not just the percentage. How to read GitHub star growth. Missing history is not zero growth.
Archive status reflects the last catalog sync, not a maintenance assessment. Check recent releases, issue responses, and fit for your project with the repository shortlisting checklist.
0 paths
Agent instructions and tool configuration found in this repository.
No config files detected.
1 observed capture since 2026-11-09. Observed captures are shown by default.
Stars from first capture 0
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.
De-AI writing skill for any Agent Skills-compatible agent (77+ via the Skills CLI), with native plugins for Claude Code, Codex, Grok Build, and Antigravity. Narrative-architecture repair for fiction, venue-matched rules for professional prose. Based on StoryScope (arXiv:2604.03136).
Natural Language Processing in Clojure (opennlp)
Unsupervised text tokenizer for Neural Network-based text generation.
The open document intelligence platform for builders and hackers - DMS for the agentic world