deepspeed vs minimind
Two of the top projects in Training and fine-tuning, side by side: who leads on each score part and who each one suits, then setup, license, activity and what each review found.
deepspeed
PyTorch library for distributed training of large models with ZeRO
minimind
Train a 64M-parameter LLM from scratch in plain PyTorch
Verdict
deepspeed ranks higher, 72 to 64. Of the 4 score parts we measure for both, deepspeed leads on 3, minimind on 1.
- Adoptionminimind leads by 33 points, 77 to 44: a wide gap.
- Freshnessdeepspeed leads by 15 points, 100 to 85: a wide gap.
- Maintenancedeepspeed leads by 30 points, 81 to 51: a wide gap.
- Agent-readydeepspeed leads by 45 points, 45 to 0: a wide gap.
Pick deepspeed if you are
ML engineers training or fine-tuning large models on multi-GPU clusters
- Used to train models such as BLOOM 176B and Megatron-Turing NLG 530B
- Integrates with Hugging Face Transformers/Accelerate, Lightning, Composer, Determined and MMEngine
Pick minimind if you are
Engineers learning LLM internals by pretraining and fine-tuning small models
- Core algorithms (LoRA, DPO, PPO/GRPO) written in plain PyTorch, not wrapped from trl or peft
- Open datasets for each stage, with mini variants (1.2GB pretrain, 1.6GB SFT) for quick runs
| What we compare | deepspeed | minimind |
|---|---|---|
| Score parts, out of 100 | ||
| Adoption | 44, known | 77, popular |
| Freshness | 100, active | 85, active |
| Maintenance | 81, healthy | 51, fair |
| Easy to run | Not scored for this kind | Not scored for this kind |
| Agent-ready | 45, minimal | 0, none |
| Facts from GitHub and the README | ||
| Stars | 43.2k | 63.7k |
| License | Apache-2.0 (permissive) | Apache-2.0 (permissive) |
| Last commit | Oct 2026 | Sep 2026 |
| Last release | Sep 2026 | Oct 2025 |
| Language | Python | Python |
| Docker | Yes | No |
| GPU | Optional | Optional |
| arm64 or Apple Silicon | Not stated | Not stated |
deepspeed
DeepSpeed is a PyTorch optimization library for training large deep learning models across many GPUs. It provides ZeRO memory partitioning with CPU and NVMe offload, 3D parallelism, Ulysses sequence parallelism and MoE support, configured through a JSON file. It integrates with Hugging Face Transformers and Accelerate, Lightning, MosaicML Composer, Determined and MMEngine.
Who it is for: ML engineers training or fine-tuning large models on multi-GPU clusters
Strengths
- Used to train models such as BLOOM 176B and Megatron-Turing NLG 530B
- Integrates with Hugging Face Transformers/Accelerate, Lightning, Composer, Determined and MMEngine
- Supports NVIDIA, AMD, Intel Gaudi/XPU/CPU, Huawei Ascend NPU and Tecorigin accelerators
- Installs from PyPI; C++/CUDA ops are JIT-compiled or pre-built
Weaknesses
- Needs PyTorch installed first and a CUDA or ROCm compiler for JIT ops
- Mainly tested on NVIDIA Pascal to Hopper and AMD MI100/MI200
- Windows lacks async IO and GDS support
- Large JSON config surface; ZeRO and offload tuning requires reading the docs
- GPU optional
- Docker
- Needs PyTorch, CUDA or ROCm compiler (nvcc/hipcc), ninja
- Models: PyTorch models, Hugging Face Transformers
minimind
MiniMind is a from-scratch implementation of a small Qwen3-style decoder-only LLM (64M dense, 198M-A64M MoE) with each training stage written in native PyTorch: pretrain, SFT, LoRA, DPO, PPO/GRPO/CISPO, agentic RL and distillation. It ships datasets, a 6,400-token tokenizer, an OpenAI-compatible server and a Streamlit chat UI. The README states one SFT epoch takes about 2 hours on a single 3090.
Who it is for: Engineers learning LLM internals by pretraining and fine-tuning small models
Strengths
- Core algorithms (LoRA, DPO, PPO/GRPO) written in plain PyTorch, not wrapped from trl or peft
- Open datasets for each stage, with mini variants (1.2GB pretrain, 1.6GB SFT) for quick runs
- Weights convert to transformers, llama.cpp, vllm and ollama; ollama and vllm usage is documented
- Checkpoint resume, DDP and DeepSpeed multi-GPU, wandb or swanlab logging
Weaknesses
- Models are tiny (64M), so output quality is far below mainstream LLMs
- README is primarily in Chinese; the English version is a separate file
- Training data is largely synthetic and distilled, with mixed source licenses
- Native MoE training is about 50% slower than dense, per the README
- GPU optional
- Needs PyTorch, transformers, streamlit (optional WebUI), swanlab or wandb (optional)
- Models: minimind-3 (64M), minimind-3-moe (198M-A64M), minimind2 series, Transformers format, llama.cpp
More in Training and fine-tuning
Building with these?
We help teams pick an open-source AI stack and run it in production. Client work never changes a rank.