#3333rd of 34 in Open models
chinese-bert-wwm
Chinese BERT and RoBERTa checkpoints pre-trained with whole word masking
- Stars
- 10.2k
- License
- Apache-2.0
- Last commit
- Apr 2026
- Language
- Python
10,140 on 31 Mar 202610,231 on 20 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Pre-trained Chinese BERT-wwm, BERT-wwm-ext, RoBERTa-wwm-ext (base and large), RBT3 and RBTL3 checkpoints from HFL. Whole word masking uses LTP word segmentation so all characters of a word are masked together. Weights are TensorFlow checkpoints (Baidu Netdisk) and PyTorch weights on Hugging Face, loadable with BertTokenizer/BertModel; the README includes benchmark tables on CMRC 2018, DRCD, XNLI and others.
Who it is for: NLP engineers fine-tuning Chinese encoders for classification, QA and matching
Strengths
- Hugging Face and PaddleHub loading, via BertTokenizer/BertModel with hfl/ model names
- Includes 3-layer RBT3 (38M) and RBTL3 (61M) models for lower-cost inference
- Benchmark tables report max and mean over 10 seeds on eight Chinese datasets
- Same vocab and config as Google's Chinese BERT, so existing BERT code works unchanged
Weaknesses
- Pre-training code is not provided; the FAQ points to issue threads only
- Released weights omit the MLM head, so masked-LM use needs further pre-training
- Original checkpoints are TensorFlow; PyTorch needs conversion or the HF download
- README is mostly Chinese, and the repo is a 2019-era encoder line, with a banner promoting newer LLM projects
What it needs
- GPU optional
- Needs transformers, TensorFlow or PyTorch, PaddleHub (optional)
- Models: BERT-wwm, BERT-wwm-ext, RoBERTa-wwm-ext, RoBERTa-wwm-ext-large, RBT3
Also in Open models
See all 34| Rank | Project | Score |
|---|---|---|
| 1 | llmfitRust tool that checks which LLMs fit your RAM, CPU and GPU | 87 out of 100 |
| 2 | layaSingle-pass multilingual classifier for typed choice, score and yes/no decisions | 84 out of 100 |
| 3 | timesfmPretrained time-series foundation model for zero-shot forecasting from Google Research | 75 out of 100 |
| #4 | speechPyTorch toolkit for training and deploying ASR, TTS and speech LLM models | 70 out of 100 |
| #5 | yueLyrics-to-song model that writes an editable melody and chord score first | 65 out of 100 |