Search awesome repositories

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

lakeFS - Data version control for your data lake | Git for data

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

PySpark + Scikit-learn = Sparkit-learn

A Spark UI and Spark History Server alternative with CPU and Memory metrics! Delight is free, cross-platform, and open-source.

Serverless proxy for Spark cluster

A pure Python implementation of Apache Spark's RDD and DStream interfaces.

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Big Data essentials: Hadoop, MapReduce, Spark. Explore tutorials and demos in Jupyter notebooks—most are self-contained and live, ready to run with a click.

Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-squares, quantile, and expectile regression.

bring sf to spark in production

An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark.