#5151st of 65 in Model serving
mlx-vlm
Run and fine-tune vision, audio and text models on Apple Silicon with MLX
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 5.8k
- License
- MIT
- Last commit
- Nov 2026
- Last release
- Nov 2026
- Language
- Python
4,401 on 21 Apr 20265,834 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
MLX-VLM is a Python package that runs multimodal models (text, image, video, audio in) on Macs via MLX. It provides a CLI (mlx_vlm.generate), a Python API, a Gradio chat UI, and a FastAPI server with continuous batching, prefix caching and KV cache quantization. It also supports speculative decoding with DFlash, MTP and EAGLE-3 drafters, model conversion and quantization, and fine-tuning.
Who it is for: Mac users running or fine-tuning local multimodal models via CLI, Python or API
Strengths
- Covers many model families with per-model docs, including OCR, Gemma 4, MiniCPM-o and Qwen VL
- Speculative decoding with DFlash, MTP and EAGLE-3 drafters; README reports up to 3.94x on Gemma 4 26B-A4B
- Server adds continuous batching, automatic prefix caching and KV cache quantization
- Includes mlx_vlm.convert for quantizing Hugging Face models, plus a fine-tuning section
Weaknesses
- Targets macOS on Apple silicon; the only CUDA mention is activation quantization
- Gradio chat UI needs the separate mlx-vlm[ui] extra
- DSpark speculative decoding requires greedy sampling (temperature=0)
- No Docker or compose files provided
What it needs
- GPU required
- Needs MLX, Hugging Face model hub
- Models: MLX, Hugging Face safetensors (converted via mlx_vlm.convert), Qwen2-VL, Qwen3.5, Gemma 4
Quick answers
Is mlx-vlm open source?
Yes. Its code is on GitHub under the MIT license.
Does mlx-vlm need a GPU?
Yes, it needs a GPU.
Also in Model serving
See all 65| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server | 94 out of 100 |
| 2 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 92 out of 100 |
| 3 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 91 out of 100 |
| #4 | vLLMInference engine serving LLMs with PagedAttention and an OpenAI-compatible API | 88 out of 100 |
| #5 | strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM | 88 out of 100 |