#5656th of 105 in Model serving
vllm-ascend
Hardware plugin that runs vLLM inference on Huawei Ascend NPUs
- Stars
- 3.1k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Oct 2026
- Language
- Python
2,001 on 29 Apr 20263,128 on 18 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Inferact is hiring: 32 open roles
Overview
vllm-ascend is a community-maintained plugin that adds a Huawei Ascend NPU backend to vLLM through its hardware-pluggable interface. It lets Transformer, Mixture-of-Experts, embedding and multi-modal models run on Atlas 800I A2/A3, Atlas 300I Duo and 950PR/950DT hardware. Releases track vLLM versions, with per-release branches and CI.
Who it is for: Teams serving LLMs with vLLM on Huawei Ascend NPU hardware
Strengths
- Uses vLLM's hardware-pluggable interface, so the core vLLM code is not forked
- Covers dense, MoE, embedding and multi-modal models on Ascend NPUs
- Release branches per vLLM version, each with a stated CI commitment
- Docs include a support matrix, quick start and large-scale expert parallelism guide
Weaknesses
- Runs only on Huawei Ascend NPUs, not on NVIDIA or AMD GPUs
- Pins exact versions: Python 3.12, CANN 9.2.1, PyTorch 2.10.0
- The vLLM version must match the plugin version
- Older release branches such as v0.13.0 receive bug fixes only
What it needs
- GPU required
- Docker
- Needs vLLM, Huawei Ascend NPU, CANN 9.2.1, PyTorch 2.10.0, TorchNPU 2.10.0.post7, Python 3.12
- Models: Transformer LLMs, Mixture-of-Experts, Embedding, Multi-modal LLMs
What changed
- v0.30.0 ↗ (opens in a new tab)Security fix
CVE-2026-114053. “There is a known security issue CVE-2026-114053 about vLLM upstream between v0.7.3 and v0.31.0, and vLLM has resolved this issue in main branch by vllm-project/vllm#58830. Please pay attention to whe…”
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Known problems
Open now, reported in the last 90 days
- [Bug]: CPU binding silently skipped on Ascend 910B3: physical NPU ID used as key but lookup uses logic ID ↗ (opens in a new tab)1 reactions
- [Bug]: DeepSeek-V4.1-Flash outputs gibberish on A3 dual-node mixed deployment with vllm-ascend 0.27.1 ↗ (opens in a new tab)1 reactions
- [Bug]: glm-5.3-w8a8c8在vllm-ascend0.26.0rc版本推理启动成功,evalscope性能测试崩溃 ↗ (opens in a new tab)1 reactions
- [Misc]: 为什么2台服务器当1D ↗ (opens in a new tab)1 reactions
- [Bug]: Dsv4 speculative dsaprk error on 910C ↗ (opens in a new tab)1 reactions
Reported, then fixed by a linked change
- [Bug]: off-stride `max_num_seqs` silently disables ACL graph capture for the largest batches (~7.5x slower decode) ↗ (opens in a new tab)1 reactions · closed 23 Sept 2026Fixed by [BugFix] Add off-stride graph capture size ceilings ↗ (opens in a new tab)
- 【BUG】CPU binding may fail where the operating system language is Chinese ↗ (opens in a new tab)1 reactions · closed 16 Apr 2026Fixed by [BugFix] Enforce C locale for CPU binding subprocess parsing ↗ (opens in a new tab)
- [Bug]: ModelSlim MoE weight-loader wrapper retains discarded layers through reload metadata ↗ (opens in a new tab)0 reactions · closed 30 Oct 2026Fixed by [BugFix][Quantization] Release ModelSlim MoE reload metadata references ↗ (opens in a new tab)
- [Bug]: Benchmark report classification depends on parent directory names ↗ (opens in a new tab)0 reactions · closed 19 Oct 2026Fixed by [Misc] Remove unused benchmark report converter script ↗ (opens in a new tab)
- [Bug]: Static kernel cache is not reused after restarting with the same model ↗ (opens in a new tab)0 reactions · closed 19 Oct 2026Fixed by [BugFix][Compilation] Initialize static kernel runtime before loading cached graphs ↗ (opens in a new tab)
Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.
Also in Model serving
See all 105| Rank | Project | Score |
|---|---|---|
| 1 | llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server | 94 out of 100 |
| 2 | OllamaRuns open models locally through a CLI and REST API | 91 out of 100 |
| 3 | LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs | 91 out of 100 |
| #4 | strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs | 89 out of 100 |
| #5 | modularMojo language and MAX inference framework with an OpenAI-compatible server | 89 out of 100 |