#7474th of 75 in ML frameworks and libraries
cleanrl
Single-file PyTorch and JAX implementations of deep RL algorithms
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 10.6k
- License
- custom license
- Last commit
- Apr 2026
- Last release
- Nov 2022
- Language
- Python
9,368 on 31 Mar 202610,584 on 20 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
CleanRL provides standalone training scripts for PPO, DQN, C51, SAC, DDPG, TD3, PPG, RND and Qdagger, each holding all implementation details in one file (ppo_atari.py is about 340 lines). Scripts log to TensorBoard and optionally Weights and Biases, support seeding and gameplay video capture, and can run on AWS Batch. The README states it is not a modular library and is not meant to be imported.
Who it is for: RL researchers and students who want readable reference implementations
Strengths
- Each algorithm variant is one self-contained file, easy to read and modify
- Benchmarked on 7+ algorithms and 34+ games with public W&B reports
- TensorBoard, W&B tracking, seeding and video capture built into the scripts
- Includes JAX variants and envpool-based Atari scripts for faster training
Weaknesses
- Not importable as a library; code is duplicated across files by design
- README states Python >=3.7.1,<3.11
- Last release is 2022-11-14; gymnasium migration noted as in progress
- License detected as NOASSERTION while README badge says MIT
What it needs
- GPU optional
- Docker
- Needs PyTorch, JAX, Gymnasium, TensorBoard, Weights and Biases, uv, envpool, AWS Batch
- Models: PPO, DQN, C51, SAC, DDPG
Also in ML frameworks and libraries
See all 75| Rank | Project | Score |
|---|---|---|
| 1 | transformersPython library defining open models for text, vision, audio and multimodal tasks | 94 out of 100 |
| 2 | pytorchPython tensor library with GPU support and tape-based autograd for deep learning | 94 out of 100 |
| 3 | ultralyticsPython library and CLI for training and running YOLO vision models | 92 out of 100 |
| #4 | pytorch-image-modelsPyTorch library of image models, pretrained weights and reference training scripts | 91 out of 100 |
| #5 | supervisionPython toolkit for loading, annotating and counting computer vision detections | 90 out of 100 |