#44th of 12 in Training and fine-tuning
verl
Reinforcement learning training library for LLM post-training with PPO and GRPO
- Stars
- 23.8k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Sep 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
verl is the open-source implementation of the HybridFlow paper from ByteDance Seed. It runs RL and supervised fine-tuning for LLMs using FSDP, FSDP2 or Megatron-LM for training and vLLM, SGLang or HF Transformers for rollout generation. It covers PPO, GRPO, RLOO, ReMax and others, with multi-turn tool calling, VLM support and scaling to 671B models.
Who it is for: ML engineers running RL post-training of LLMs on GPU clusters
Strengths
- Pluggable backends: FSDP/FSDP2/Megatron-LM for training, vLLM/SGLang/HF for rollout
- Many RL algorithms built in: PPO, GRPO, ReMax, RLOO, REINFORCE++
- Scales to 671B models and hundreds of GPUs via expert parallelism
- Supports NVIDIA, AMD and Ascend hardware; LoRA RL to save memory
Weaknesses
- Needs multi-GPU clusters in practice; minimum hardware not stated in README
- Many algorithm recipes live in a separate verl-recipe repository and submodule
- Version-sensitive: README warns against vLLM 0.7.x and requires >=0.8.2
- Several features remain under verl/experimental until merged into the main library
What it needs
- GPU required
- Docker
- Needs Ray, PyTorch, vLLM, SGLang, Megatron-LM
- Models: Qwen3.5, Qwen3, Qwen2.5, Qwen2.5-VL, Kimi-VL
Head to head
Also in Training and fine-tuning
See all 12| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run, fine-tune and export open models | 88 out of 100 |
| 2 | deepspeedPyTorch library for distributed training of large models with ZeRO | 79 out of 100 |
| 3 | llamafactoryFine-tuning toolkit for 100+ LLMs with a CLI and Gradio web UI | 74 out of 100 |
| #5 | minimindTrain a 64M-parameter LLM from scratch in plain PyTorch | 66 out of 100 |
| #6 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 65 out of 100 |