#6565th of 68 in Training and fine-tuning
flagai
PyTorch toolkit for training and fine-tuning Chinese and English language models
- Stars
- 3.9k
- License
- Apache-2.0
- Last commit
- Jul 2026
- Last release
- Nov 2024
- Language
- Python
3,849 on 25 Apr 20263,864 on 14 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
FlagAI is a Python toolkit from BAAI that loads pre-trained models through an AutoLoader and Predictor API, then trains or fine-tunes them with PyTorch, DeepSpeed, Megatron-LM or BMTrain. It covers over 30 models, including Aquila, AltCLIP, AltDiffusion, GLM, OPT, BERT, GPT2 and T5, with a focus on Chinese text tasks such as NER, title generation and semantic matching.
Who it is for: ML engineers fine-tuning GLM, BERT or CLIP models, especially on Chinese text
Strengths
- Switches between PyTorch, DeepSpeed, Megatron-LM and BMTrain parallelism in under ten lines of code
- AutoLoader and Predictor give one API for NER, seq2seq, title generation and semantic matching
- Model table lists train, finetune and inference support per model, with example directories
- Includes BMInf for low-resource inference and Flash Attention for BERT and ViT training
Weaknesses
- Model zoo is mostly 2022-era models (GLM, OPT, T5, BERT); no recent LLM families listed
- README news ends at v1.7.0 (June 2023); last GitHub release is 2024-11-07
- Mixed licenses: Megatron-LM, GLM (MIT) and AltDiffusion (Open RAIL-M) differ from Apache-2.0
- Some example links in the toolkit table are malformed (http:/// URLs)
What it needs
- GPU optional
- Docker
- Needs Python >= 3.8, PyTorch >= 1.8.0, DeepSpeed >= 0.7.7 (optional), BMTrain >= 0.2.2 (optional), BMInf (optional), NVIDIA apex (optional), flash-attn >= 1.0.2 (optional), CUDA and NCCL (optional, for GPU)
- Models: Aquila, AltCLIP, AltDiffusion, GLM, EVA-CLIP
What changed
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Also in Training and fine-tuning
See all 68| Rank | Project | Score |
|---|---|---|
| 1 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 89 out of 100 |
| 2 | unslothDesktop and web app to run and fine-tune local models | 88 out of 100 |
| 3 | trlPost-training library with SFT, DPO, GRPO and KTO trainers | 88 out of 100 |
| #4 | deepspeedPyTorch library for distributed training of very large models with ZeRO | 87 out of 100 |
| #5 | megatron-lmDistributed training library for large transformer models on NVIDIA GPUs | 86 out of 100 |