ArchestackBest of Open-Source AI: List on GitHub ↗ (opens in a new tab)

#4848th of 100 in Learning and cookbooks

llms-from-scratch-cn

Chinese notebooks for building GPT, Llama3, RWKV and ChatGLM from scratch

Repository on GitHub ↗ (opens in a new tab)
Stars
4.4k
License
custom license
Last commit
Sep 2026
Language
Jupyter Notebook

Missed the latest GitHub check; these facts may be older.

Overview

Datawhale tutorial repo that walks through implementing a GPT-style LLM in PyTorch notebooks, based on rasbt/LLMs-from-scratch with a Chinese translation. It covers text processing, attention, the GPT model and pretraining on unlabeled data, plus architecture notebooks for ChatGLM3/4, Llama3, RWKV V2 to V6 and MiniCPM. Chapters on fine-tuning and human feedback are still marked as coming soon.

Who it is for: Developers and researchers learning LLM internals in Chinese

Strengths

  • Runnable notebooks with exercise solutions for chapters 2-5 and appendices A and D
  • Architecture walkthroughs for ChatGLM3/4, Llama3, MiniCPM and RWKV V2 to V6
  • Needs only PyTorch basics; includes a PyTorch intro appendix with a DDP script
  • Offers a condensed Codes track and a detailed Translated_Book track

Weaknesses

  • Chapters 6-8 (classification fine-tuning, human feedback, practical use) are marked coming soon
  • Text is Chinese only; core content derives from rasbt's English original
  • License is unclear: README shows both an Apache-2.0 badge and a CC BY-NC-SA 4.0 notice
  • Teaches small educational models; not a training or serving stack

What it needs

  • Needs PyTorch, Jupyter
  • Models: GPT, Llama3, ChatGLM3, ChatGLM4, RWKV V2-V6

Also in Learning and cookbooks

See all 100
Also in Learning and cookbooks
RankProjectScore
1ai-engineering-from-scratchHands-on AI engineering curriculum with 542 lessons across 20 phases78.6k stars+10,601 (+16%) in 30 daysMIT91 out of 100
2generative-ai-for-beginners21-lesson course on building generative AI apps in Python and TypeScript123.4k stars+2,358 (+1.9%) in 30 daysMIT83 out of 100
3rag_techniquesCollection of 42 runnable notebooks implementing RAG techniques30.3k stars+550 (+1.8%) in 30 dayscustom license82 out of 100
#4cs249r_bookOpen textbook and course materials for machine learning systems engineering30.2k stars+1,272 (+4.4%) in 30 dayscustom license82 out of 100
#5ai-for-beginners24-lesson AI curriculum with notebooks in PyTorch and TensorFlow Live demo ↗ (opens in a new tab)70.7k stars+1,148 (+1.6%) in 30 daysMIT81 out of 100