#3434th of 41 in Training and fine-tuning
rllm
RL training framework for language agents, on any harness or sandbox
- Stars
- 5.9k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Apr 2026
- Language
- Python
5,325 on 3 Apr 20265,905 on 30 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
rLLM trains language agents with reinforcement learning (GRPO, REINFORCE, RLOO, SFT, on-policy distillation). A model gateway captures token IDs and logprobs from your existing agent code, so the same agent runs for eval and training. Backends are verl, tinker and fireworks, chosen with one flag, and 60+ benchmarks run through `rllm eval`.
Who it is for: Researchers and engineers doing RL post-training of LLM agents
Strengths
- Same agent code for eval and training; gateway captures token IDs and logprobs
- Switches between verl, tinker and fireworks backends with one flag
- 60+ benchmarks, including SWE-bench and Terminal-Bench 2.0, via `rllm eval`
- Wraps LangGraph, OpenAI Agents SDK or plain OpenAI client code with @rllm.rollout
Weaknesses
- Default install targets the tinker backend; multi-GPU needs the verl extra
- README installs from a git URL, not from PyPI
- Training recipes and results live mostly in external blog posts
What it needs
- GPU optional
- Docker
- Needs Python >= 3.11, verl with vLLM or SGLang (multi-GPU), Tinker API, Fireworks, Docker, Daytona or Modal (sandboxes)
- Models: OpenAI-compatible endpoints, vLLM, SGLang
Quick answers
Is rllm open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Can you run rllm with Docker?
Yes. Its repository has a Docker setup: a Dockerfile, a Compose file or a published image.
Does rllm need a GPU?
No. It runs on a CPU and can use a GPU when you have one.
Also in Training and fine-tuning
See all 41| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 2 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 86 out of 100 |
| 3 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 85 out of 100 |
| #4 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 84 out of 100 |
| #5 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 83 out of 100 |