#1717th of 23 in Open models
needle
8-29 MB on-device model for tool calling, extraction and embeddings
Live demo ↗
(opens in a new tab)Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 13.6k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Language
- Python
Missed the latest GitHub check; these facts may be older.
Overview
Needle is a small model from Cactus Compute that picks functions and fills their arguments from user text, extracts typed fields from messy text, and returns sentence embeddings. A grammar compiled from your schemas constrains decoding, and each response carries a confidence score. It ships with a C++ engine for many platforms, a Python package, and the Whistle speech-to-text model that runs on the same engine.
Who it is for: Developers adding on-device tool calling to mobile, edge and embedded apps
Strengths
- Whole model is an 8-29 MB file, with depths from 2 to 20 layers deployable
- Grammar-constrained decoding guarantees output parses against your schemas
- Returns empty call list for off-topic requests, plus a calibrated confidence score
- Local LoRA fine-tuning or hosted full fine-tuning from the same package
Weaknesses
- Trades general chat capacity for tool calls and extraction; not a chat model
- Full fine-tuning runs on Cactus GPUs and needs an API key
- Binary telemetry is on by default; opt out via environment variables
- JavaScript/WASM package is community-maintained, not first-party
What it needs
- no GPU
- Needs Python (pip install cactus-needle), JAX (for local fine-tuning)
- Models: Needle 3 (.cact), Needle 2, Whistle speech-to-text (.cact)
Also in Open models
See all 23| Rank | Project | Score |
|---|---|---|
| 1 | llmfitRust tool that checks which LLMs fit your RAM, CPU and GPU | 86 out of 100 |
| 2 | layaSingle-pass multilingual classifier for typed choice, score and yes/no decisions | 83 out of 100 |
| 3 | timesfmPretrained time-series foundation model for zero-shot forecasting from Google Research | 74 out of 100 |
| #4 | unlimited-ocrOCR model that parses long documents and multi-page PDFs in one pass | 69 out of 100 |
| #5 | speechPyTorch toolkit for training and deploying ASR, TTS and speech LLM models | 67 out of 100 |