#5959th of 70 in Open models
rex-omni
3B multimodal model for detection, pointing, OCR and keypoints
Live demo ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 1.6k
- License
- custom license
- Last commit
- Feb 2026
- Language
- Jupyter Notebook
1,299 on 21 Apr 20261,638 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
Rex-Omni is a 3B-parameter multimodal LLM, based on Qwen2.5-VL, that emits boxes, points, polygons and keypoints as text tokens. The repo provides a Python wrapper with transformers and vLLM backends, tutorials for each task, a Gradio demo, evaluation code, and SFT/GRPO fine-tuning for pointing. An AWQ-quantized checkpoint is also published.
Who it is for: Computer vision engineers needing promptable detection, pointing and OCR
Strengths
- One model covers detection, pointing, visual prompting, OCR boxes/polygons, keypoints and GUI grounding
- Two backends: transformers for simplicity, vLLM for batch throughput
- AWQ checkpoint published, which the README says halves storage
- Includes evaluation code, fine-tuning code, and a SAM pairing for masks
Weaknesses
- Custom IDEA License 1.0 on a Qwen research-licensed base; not a standard OSI license
- Install pins torch 2.7.0 with CUDA 12.8 wheels, so it targets NVIDIA GPUs
- Gradio demo default server_name is a hardcoded private IP and needs overriding
- No release tags and no Docker setup
What it needs
- GPU required
- Needs PyTorch 2.7.0, CUDA 12.8, transformers, vLLM (optional), Gradio
- Models: Rex-Omni 3B, Rex-Omni-AWQ, Hugging Face checkpoints
- port 5211
Quick answers
Is rex-omni open source?
Its code is public on GitHub under its own license terms; read them before you use it at work.
Does rex-omni need a GPU?
Yes, it needs a GPU.
Also in Open models
See all 70| Rank | Project | Score |
|---|---|---|
| 1 | ruviewWiFi CSI sensing on ESP32 for presence, breathing, heart rate and pose | 94 out of 100 |
| 2 | layaSingle-pass multilingual classifier for typed choice, score and yes/no decisions | 88 out of 100 |
| 3 | rf-detrReal-time transformer models for object detection, segmentation and keypoints | 79 out of 100 |
| #4 | timesfmPretrained time-series foundation model for zero-shot forecasting from Google Research | 78 out of 100 |
| #5 | yolov5PyTorch YOLOv5 object detection, segmentation and classification models with training code | 76 out of 100 |