minimind vs nanochat
Two of the top projects in Training and fine-tuning, side by side: who leads on each score part and who each one suits, then setup, license, activity and what each review found.
minimind
Train a 64M-parameter LLM from scratch in plain PyTorch
nanochat
Minimal PyTorch codebase to train and chat with a small GPT-2-class LLM
Verdict
minimind ranks higher, 64 to 49. Of the 4 score parts we measure for both, minimind leads on 2, nanochat on 2.
- Adoptionminimind leads by 15 points, 77 to 62: a wide gap.
- Freshnessnanochat leads by 10 points, 95 to 85.
- Maintenanceminimind leads by 49 points, 51 to 2: a wide gap.
- Agent-readynanochat leads by 15 points, 15 to 0: a wide gap.
Pick minimind if you are
Engineers learning LLM internals by pretraining and fine-tuning small models
- Core algorithms (LoRA, DPO, PPO/GRPO) written in plain PyTorch, not wrapped from trl or peft
- Open datasets for each stage, with mini variants (1.2GB pretrain, 1.6GB SFT) for quick runs
Pick nanochat if you are
Researchers and engineers learning or experimenting with end-to-end LLM training
- Covers tokenizer, pretraining, SFT, RL, eval and inference in one repo
- Single `--depth` dial sets all other hyperparameters automatically
| What we compare | minimind | nanochat |
|---|---|---|
| Score parts, out of 100 | ||
| Adoption | 77, popular | 62, popular |
| Freshness | 85, active | 95, active |
| Maintenance | 51, fair | 2, weak |
| Easy to run | Not scored for this kind | Not scored for this kind |
| Agent-ready | 0, none | 15, none |
| Facts from GitHub and the README | ||
| Stars | 63.7k | 58.6k |
| License | Apache-2.0 (permissive) | MIT (permissive) |
| Last commit | Sep 2026 | Jul 2026 |
| Last release | Oct 2025 | None published |
| Language | Python | Python |
| Docker | No | No |
| GPU | Optional | Required |
| arm64 or Apple Silicon | Not stated | Mentioned |
minimind
MiniMind is a from-scratch implementation of a small Qwen3-style decoder-only LLM (64M dense, 198M-A64M MoE) with each training stage written in native PyTorch: pretrain, SFT, LoRA, DPO, PPO/GRPO/CISPO, agentic RL and distillation. It ships datasets, a 6,400-token tokenizer, an OpenAI-compatible server and a Streamlit chat UI. The README states one SFT epoch takes about 2 hours on a single 3090.
Who it is for: Engineers learning LLM internals by pretraining and fine-tuning small models
Strengths
- Core algorithms (LoRA, DPO, PPO/GRPO) written in plain PyTorch, not wrapped from trl or peft
- Open datasets for each stage, with mini variants (1.2GB pretrain, 1.6GB SFT) for quick runs
- Weights convert to transformers, llama.cpp, vllm and ollama; ollama and vllm usage is documented
- Checkpoint resume, DDP and DeepSpeed multi-GPU, wandb or swanlab logging
Weaknesses
- Models are tiny (64M), so output quality is far below mainstream LLMs
- README is primarily in Chinese; the English version is a separate file
- Training data is largely synthetic and distilled, with mixed source licenses
- Native MoE training is about 50% slower than dense, per the README
- GPU optional
- Needs PyTorch, transformers, streamlit (optional WebUI), swanlab or wandb (optional)
- Models: minimind-3 (64M), minimind-3-moe (198M-A64M), minimind2 series, Transformers format, llama.cpp
nanochat
nanochat covers tokenizer training, pretraining, SFT, RL, evaluation and a CLI chat interface in one small Python codebase. A single `--depth` setting derives width, heads, learning rate and training horizon, so one knob produces a compute-optimal model series. The reference speedrun trains a GPT-2-grade model in about 1.65 hours on an 8xH100 node per the leaderboard.
Who it is for: Researchers and engineers learning or experimenting with end-to-end LLM training
Strengths
- Covers tokenizer, pretraining, SFT, RL, eval and inference in one repo
- Single `--depth` dial sets all other hyperparameters automatically
- Public time-to-GPT-2 leaderboard with commits and CORE scores
- Same scripts run on one GPU via gradient accumulation, or on CPU/MPS
Weaknesses
- Reference speedrun needs an 8xH100 node, roughly $48 of rented compute
- Under 80GB VRAM requires manually lowering `--device-batch-size`
- CPU/MPS path only trains a tiny model with weak results
- Chat is CLI-only in the README; resulting model is small and hallucinates
- GPU required
- Needs PyTorch, uv, wandb
- Models: nanochat GPT (depth-scaled), bfloat16, float32, float16
More in Training and fine-tuning
Building with these?
We help teams pick an open-source AI stack and run it in production. Client work never changes a rank.