Awesome Opensource Ai

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

Point Cloud Library (PCL)

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Python bindings for llama.cpp

High-speed Large Language Model Serving for Local Deployment

cuDF - GPU DataFrame Library

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Easy-to-use image segmentation library with awesome pre-trained model zoo, supporting wide-range of practical tasks in Semantic Segmentation, Interactive Segmentation, Panoptic Segmentation, Image Matting, 3D Segmentation, etc.

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

Accessible large language models via k-bit quantization for PyTorch.

An app that brings language models directly to your phone.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Flax is a neural network library for JAX that is designed for flexibility.

lightweight, standalone C++ inference engine for Google's Gemma models.

the portable Python dataframe library

Efficient Triton Kernels for LLM Training

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.

A query and indexing engine for Redis, providing secondary indexing, full-text search, vector similarity search and aggregations.

A treasure chest for visual classification and recognition powered by PaddlePaddle

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.

Header-only C++/python library for fast approximate nearest neighbors

Local-first healthcare AI: clinical NER & HIPAA PII de-identification that runs 100% on-device. 2,200+ medical models, 21 languages, Apple MLX + Python, no cloud, no patient data leaving your network. Apache-2.0

cuML - RAPIDS Machine Learning Library

On-device AI across mobile, embedded and edge for PyTorch

A distributed approximate nearest neighborhood search (ANN) library which provides a high quality vector index build, search and distributed online serving toolkits for large scale vector search scenario.

An Engine-Agnostic Deep Learning Framework in Java

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

Fast inference engine for Transformer models

Pre-trained Deep Learning models and demos (high quality and extremely fast)

RTAB-Map library and standalone application

A retargetable MLIR-based machine learning compiler and runtime toolkit.

A flexible, high-performance 3D simulator for Embodied AI research.

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.

llama.cpp fork with additional SOTA quants and improved performance

Open source, local, and self-hosted Amazon Echo/Google Home competitive Voice Assistant alternative

PyTorch native quantization for training and inference

Data manipulation and transformation for audio signal processing, powered by PyTorch

The pretty much "official" DSPy framework for Typescript

Community maintained hardware plugin for vLLM on Ascend

nanoflann: a C++11 header-only library for Nearest Neighbor (NN) search with KD-trees

Mirage Persistent Kernel: Compiling LLMs into a MegaKernel

cuGraph - RAPIDS Graph Analytics Library

Production-grade 3D gaussian splatting with CPU/GPU support for Windows, Mac and Linux 🚀