#117117th of 206 in ML frameworks and libraries
synapseml
Distributed ML pipelines on Apache Spark with LightGBM, Vowpal Wabbit and ONNX
- Stars
- 5.3k
- License
- MIT
- Last commit
- Nov 2026
- Last release
- Apr 2026
- Language
- Scala
5,193 on 21 Apr 20265,255 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
SynapseML is a Scala library, with Python, R, Java and .NET bindings, that adds distributed ML estimators to Apache Spark and shares the SparkML API. It includes LightGBM, Vowpal Wabbit, ONNX inference, Azure AI service transformers, HTTP-on-Spark, Isolation Forest, and Spark Serving for exposing Spark computations as web services. Separate Maven builds exist for Spark 3.5 (Scala 2.12) and Spark 4.0/4.1 (Scala 2.13).
Who it is for: Data engineers running ML pipelines on Spark clusters
Strengths
- Same API as SparkML, so models drop into existing Spark pipelines
- Bundles LightGBM, Vowpal Wabbit, ONNX, Isolation Forest and model interpretability tools
- Preinstalled in Microsoft Fabric; documented for Databricks, Synapse, EMR and spark-submit
- Separate builds for Spark 3.5, 4.0 and 4.1
Weaknesses
- Requires a Spark cluster or local Spark; not usable standalone
- PyPI package lacks JVM jars; mismatched Scala artifacts cause missing-class errors
- Spark 4 artifacts need a custom Maven repository on Azure blob storage
- Master targets Spark 3.5 only; Spark 4 lives on separate branches
What it needs
- GPU optional
- Needs Apache Spark 3.5 / 4.0 / 4.1, Maven repository mmlspark.blob.core.windows.net/maven
- Models: LightGBM, Vowpal Wabbit, ONNX, SparkML
What changed
- v1.1.3 ↗ (opens in a new tab)Security fix
CVE-2023-44487. “ci: fix security issues in reopen-issue-on-comment workflow by @BrendanWalsh in https://github.com/microsoft/SynapseML/pull/2502”
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Also in ML frameworks and libraries
See all 206| Rank | Project | Score |
|---|---|---|
| 1 | transformersPython library defining open models for text, vision, audio and multimodal tasks | 94 out of 100 |
| 2 | ultralyticsPython library and CLI for training and running YOLO vision models | 94 out of 100 |
| 3 | pytorchPython tensor library with GPU support and tape-based autograd | 93 out of 100 |
| #4 | pytorch-image-modelsPyTorch library of image models, pretrained weights and training scripts | 93 out of 100 |
| #5 | supervisionPython toolkit for loading, annotating and counting computer vision detections | 92 out of 100 |