#4646th of 47 in Model serving
chatrwkv
Reference inference code and chat demos for RWKV RNN language models
Live demo ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 9.5k
- License
- Apache-2.0
- Last commit
- Jul 2026
- Language
- Python
9,435 on 1 Apr 20269,495 on 21 Oct 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
ChatRWKV is the Python reference implementation for running RWKV, an RNN-based language model, as a chatbot. It ships the `rwkv` pip package, a v2 runtime with "stream" and "split" strategies and INT8 (the README says 3 GB VRAM is enough for RWKV 14B), an optional CUDA kernel, and short model files for building your own engine. It is a developer reference rather than a packaged chat app.
Who it is for: Developers building or studying RWKV inference and chat code
Strengths
- Stream, split and INT8 strategies fit large models into small VRAM
- RNN state lets you carry context forward without re-processing the prompt
- Short readable reference files such as RWKV_in_150_lines.py and src/model_run.py
- Published as the `rwkv` pip package, with links to GGUF and HF-compatible weights
Weaknesses
- No built-in web UI or HTTP API; the README points to other projects for those
- Optional CUDA kernel needs a manual build with ninja and a CUDA toolchain
- You must track the text matching the RNN state yourself to avoid bugs
- README is mostly links and older results; much of the work lives in other repos
What it needs
- GPU optional
- Needs PyTorch, ninja (for CUDA kernel), CUDA toolkit (optional)
- Models: RWKV-4 Raven, RWKV World (v4/v5/v6), RWKV-7, PyTorch .pth, GGUF (via linked weights)
Also in Model serving
See all 47| Rank | Project | Score |
|---|---|---|
| 1 | LocalAISelf-hosted inference server with OpenAI-compatible API and pluggable backends | 83 out of 100 |
| 2 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 81 out of 100 |
| 3 | OllamaLocal model runner with CLI and REST API for open-weight LLMs | 80 out of 100 |
| #4 | strataRuns a 125B-parameter MoE model on one consumer GPU plus system RAM | 79 out of 100 |
| #5 | vLLMInference engine for Hugging Face LLMs with PagedAttention and continuous batching | 77 out of 100 |