#5656th of 105 in Model serving

vllm-ascend

Hardware plugin that runs vLLM inference on Huawei Ascend NPUs

Releases and problems feed (RSS)

Stars
3.1k
License
Apache-2.0
Last commit
Nov 2026
Last release
Oct 2026
Language
Python
Stars, last 29 weeks+1,127
29 Apr 2026: 2,001 stars6 May 2026: 2,019 stars13 May 2026: 2,064 stars20 May 2026: 2,098 stars27 May 2026: 2,133 stars3 Jun 2026: 2,176 stars10 Jun 2026: 2,215 stars17 Jun 2026: 2,245 stars24 Jun 2026: 2,275 stars1 Jul 2026: 2,322 stars8 Jul 2026: 2,365 stars15 Jul 2026: 2,412 stars22 Jul 2026: 2,448 stars29 Jul 2026: 2,505 stars5 Aug 2026: 2,547 stars12 Aug 2026: 2,596 stars19 Aug 2026: 2,649 stars26 Aug 2026: 2,700 stars2 Sept 2026: 2,730 stars9 Sept 2026: 2,771 stars16 Sept 2026: 2,819 stars23 Sept 2026: 2,865 stars30 Sept 2026: 2,899 stars7 Oct 2026: 2,913 stars14 Oct 2026: 2,969 stars21 Oct 2026: 2,998 stars28 Oct 2026: 3,032 stars4 Nov 2026: 3,055 stars11 Nov 2026: 3,082 stars18 Nov 2026: 3,128 stars

2,001 on 29 Apr 20263,128 on 18 Nov 2026

From GitHub's daily star counts.

Checked on GitHub.

Inferact is hiring: 32 open roles

Overview

vllm-ascend is a community-maintained plugin that adds a Huawei Ascend NPU backend to vLLM through its hardware-pluggable interface. It lets Transformer, Mixture-of-Experts, embedding and multi-modal models run on Atlas 800I A2/A3, Atlas 300I Duo and 950PR/950DT hardware. Releases track vLLM versions, with per-release branches and CI.

Who it is for: Teams serving LLMs with vLLM on Huawei Ascend NPU hardware

Strengths

  • Uses vLLM's hardware-pluggable interface, so the core vLLM code is not forked
  • Covers dense, MoE, embedding and multi-modal models on Ascend NPUs
  • Release branches per vLLM version, each with a stated CI commitment
  • Docs include a support matrix, quick start and large-scale expert parallelism guide

Weaknesses

  • Runs only on Huawei Ascend NPUs, not on NVIDIA or AMD GPUs
  • Pins exact versions: Python 3.12, CANN 9.2.1, PyTorch 2.10.0
  • The vLLM version must match the plugin version
  • Older release branches such as v0.13.0 receive bug fixes only

What it needs

  • GPU required
  • Docker
  • Needs vLLM, Huawei Ascend NPU, CANN 9.2.1, PyTorch 2.10.0, TorchNPU 2.10.0.post7, Python 3.12
  • Models: Transformer LLMs, Mixture-of-Experts, Embedding, Multi-modal LLMs

What changed

  1. v0.30.0 ↗ (opens in a new tab)Security fix

    CVE-2026-114053. “There is a known security issue CVE-2026-114053 about vLLM upstream between v0.7.3 and v0.31.0, and vLLM has resolved this issue in main branch by vllm-project/vllm#58830. Please pay attention to whe…”

From the release notes. No flag means the note does not say so, not that the upgrade is safe.

Known problems

Open now, reported in the last 90 days

Reported, then fixed by a linked change

Titles are the reporters' own words, taken from the project's GitHub issues and ranked by reactions. We have not tried to reproduce any of them.

Also in Model serving

See all 105
Also in Model serving
RankProjectScore
1llama.cppC/C++ inference engine for GGUF LLMs and VLMs with an OpenAI-compatible server135.9k starsMIT94 out of 100
2OllamaRuns open models locally through a CLI and REST API187.8k starsMITknown advisory in latest version91 out of 100
3LocalAISelf-hosted model server with OpenAI, Anthropic and ElevenLabs-compatible APIs50.5k starsMIT91 out of 100
#4strataDesktop and terminal app that runs GGUF models locally with OpenAI-compatible APIs46.4k stars+16,488 (+55%) in 30 daysMIT89 out of 100
#5modularMojo language and MAX inference framework with an OpenAI-compatible server30.2k stars+228 (+0.8%) in 30 dayscustom license89 out of 100