#2020th of 41 in Training and fine-tuning
torchtitan
PyTorch-native training stack for pretraining, fine-tuning and RL of large models
- Stars
- 5.9k
- License
- BSD-3-Clause
- Last commit
- Nov 2026
- Last release
- Oct 2026
- Language
- Python
5,158 on 3 Apr 20265,868 on 30 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
torchtitan is a PyTorch codebase for training large generative models, covering pretraining, SFT/LoRA and RL (TitanRL). It composes FSDP2, tensor, pipeline, context and expert parallelism through DeviceMesh and DTensor, and runs are defined as Python recipe functions. It supports Llama 3, Qwen3, DeepSeek V3, gpt-oss, Kimi and FLUX, with MXFP8 and NVFP4 low-precision training.
Who it is for: ML engineers and researchers training large models on multi-GPU clusters
Strengths
- Stacks FSDP2, TP, PP, CP and EP into N-D layouts without rewriting the model
- Config is a Python recipe function, so runs are readable and copyable
- TitanRL shares model definitions and kernels with vLLM, with a bitwise-parity mode
- Reads and writes Hugging Face safetensors; async distributed checkpointing
Weaknesses
- Needs a recent PyTorch nightly; stable releases pin nightly torch and torchao
- Verified recipes exist for only some models; others have CI debug configs only
- Low-precision and RL features need extra nightly installs (torchao, vLLM, Monarch)
- Aimed at multi-GPU clusters; README shows only a debug model on one GPU
What it needs
- GPU required
- Needs PyTorch nightly, torchao (optional), vLLM (RL), Monarch (RL), TorchStore (RL)
- Models: Llama 3, Qwen3, Qwen3.5, DeepSeek V3, gpt-oss
Quick answers
Is torchtitan open source?
Yes. Its code is on GitHub under the BSD-3-Clause license.
Does torchtitan need a GPU?
Yes, it needs a GPU.
Also in Training and fine-tuning
See all 41| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 2 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 86 out of 100 |
| 3 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 85 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 84 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 83 out of 100 |