#1616th of 19 in Training and fine-tuning
ms-swift
Fine-tuning, RL, quantization and serving toolkit for 600+ text and 400+ multimodal LLMs
Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 15.8k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Last release
- Sep 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
ms-swift is a Python framework from the ModelScope community for training, evaluating, quantizing and serving LLMs and multimodal models through the `swift` CLI, a Gradio Web-UI or Python. It covers LoRA/QLoRA and full-parameter SFT, DPO/KTO and GRPO-family reinforcement learning, with Megatron parallelism (TP/PP/CP/EP) for large and MoE models. Inference and deployment run on Transformers, vLLM, SGLang or LMDeploy, and evaluation uses EvalScope.
Who it is for: ML engineers fine-tuning open LLMs and multimodal models on their own GPUs
Strengths
- Covers SFT, DPO/KTO/RM, GRPO variants, embedding and reranker training in one CLI
- Megatron TP/PP/CP/EP support for full-parameter and LoRA training of MoE models
- Runs on NVIDIA CUDA, AMD ROCm, Ascend NPU, Apple MPS and CPU
- Quick start LoRA-tunes Qwen3-4B in about 13GB on a single 3090
Weaknesses
- Large dependency matrix (torch, transformers, peft, trl, deepspeed, vllm) with version pins
- No Docker image or compose file in the README
- Downloads models and datasets from ModelScope by default; Hugging Face needs --use_hf true
- 4.x/3.x split; the older 3.x line requires checking out a release branch
What it needs
- GPU optional
- Needs PyTorch, transformers, peft, modelscope, datasets, DeepSpeed (training), vLLM/SGLang/LMDeploy (inference), EvalScope (evaluation), Gradio (Web-UI)
- Models: Qwen3, Qwen3-VL, Llama4, DeepSeek-R1, GLM4.5
Also in Training and fine-tuning
See all 19| Rank | Project | Score |
|---|---|---|
| 1 | unslothDesktop and web app to run, fine-tune and export open models | 88 out of 100 |
| 2 | deepspeedPyTorch library for distributed training of large models with ZeRO | 82 out of 100 |
| 3 | verlReinforcement learning training library for LLM post-training with PPO and GRPO | 77 out of 100 |
| #4 | llamafactoryFine-tuning toolkit for 100+ LLMs with a CLI and Gradio web UI | 75 out of 100 |
| #5 | peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models | 74 out of 100 |