#6565th of 68 in Training and fine-tuning

flagai

PyTorch toolkit for training and fine-tuning Chinese and English language models

Stars
3.9k
License
Apache-2.0
Last commit
Jul 2026
Last release
Nov 2024
Language
Python
Stars, last 29 weeks+15
25 Apr 2026: 3,849 stars2 May 2026: 3,849 stars9 May 2026: 3,849 stars16 May 2026: 3,851 stars23 May 2026: 3,851 stars30 May 2026: 3,851 stars6 Jun 2026: 3,851 stars13 Jun 2026: 3,852 stars20 Jun 2026: 3,852 stars27 Jun 2026: 3,852 stars4 Jul 2026: 3,853 stars11 Jul 2026: 3,856 stars18 Jul 2026: 3,856 stars25 Jul 2026: 3,857 stars1 Aug 2026: 3,857 stars8 Aug 2026: 3,859 stars15 Aug 2026: 3,859 stars22 Aug 2026: 3,859 stars29 Aug 2026: 3,859 stars5 Sept 2026: 3,860 stars12 Sept 2026: 3,861 stars19 Sept 2026: 3,861 stars26 Sept 2026: 3,862 stars3 Oct 2026: 3,863 stars10 Oct 2026: 3,863 stars17 Oct 2026: 3,864 stars24 Oct 2026: 3,864 stars31 Oct 2026: 3,864 stars7 Nov 2026: 3,864 stars14 Nov 2026: 3,864 stars

3,849 on 25 Apr 20263,864 on 14 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Overview

FlagAI is a Python toolkit from BAAI that loads pre-trained models through an AutoLoader and Predictor API, then trains or fine-tunes them with PyTorch, DeepSpeed, Megatron-LM or BMTrain. It covers over 30 models, including Aquila, AltCLIP, AltDiffusion, GLM, OPT, BERT, GPT2 and T5, with a focus on Chinese text tasks such as NER, title generation and semantic matching.

Who it is for: ML engineers fine-tuning GLM, BERT or CLIP models, especially on Chinese text

Strengths

  • Switches between PyTorch, DeepSpeed, Megatron-LM and BMTrain parallelism in under ten lines of code
  • AutoLoader and Predictor give one API for NER, seq2seq, title generation and semantic matching
  • Model table lists train, finetune and inference support per model, with example directories
  • Includes BMInf for low-resource inference and Flash Attention for BERT and ViT training

Weaknesses

  • Model zoo is mostly 2022-era models (GLM, OPT, T5, BERT); no recent LLM families listed
  • README news ends at v1.7.0 (June 2023); last GitHub release is 2024-11-07
  • Mixed licenses: Megatron-LM, GLM (MIT) and AltDiffusion (Open RAIL-M) differ from Apache-2.0
  • Some example links in the toolkit table are malformed (http:/// URLs)

What it needs

  • GPU optional
  • Docker
  • Needs Python >= 3.8, PyTorch >= 1.8.0, DeepSpeed >= 0.7.7 (optional), BMTrain >= 0.2.2 (optional), BMInf (optional), NVIDIA apex (optional), flash-attn >= 1.0.2 (optional), CUDA and NCCL (optional, for GPU)
  • Models: Aquila, AltCLIP, AltDiffusion, GLM, EVA-CLIP

What changed

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Also in Training and fine-tuning

See all 68
Also in Training and fine-tuning
RankProjectScore
1peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models Live demo ↗ (opens in a new tab)21.9k starsApache-2.089 out of 100
2unslothDesktop and web app to run and fine-tune local models Live demo ↗ (opens in a new tab)79.7k starsApache-2.088 out of 100
3trlPost-training library with SFT, DPO, GRPO and KTO trainers19.8k starsApache-2.088 out of 100
#4deepspeedPyTorch library for distributed training of very large models with ZeRO43.3k starsApache-2.087 out of 100
#5megatron-lmDistributed training library for large transformer models on NVIDIA GPUs18.4k starscustom license86 out of 100