#5252nd of 70 in Open models
cogagent
Open 9B vision-language model that reads screenshots and outputs GUI actions
Live demo ↗
(opens in a new tab)Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 1.2k
- License
- Apache-2.0
- Last commit
- Oct 2026
- Language
- Python
1,155 on 21 Apr 20261,198 on 10 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Overview
CogAgent-9B-20241220 is a GLM-4V-9B-based vision-language model that takes a screenshot, a task and step history, and returns a grounded GUI operation such as CLICK with a bounding box. It supports Windows, macOS and Android screens in Chinese and English. The repo provides transformers inference scripts, a Gradio-style web demo, and SFT/LoRA fine-tuning code.
Who it is for: Researchers and developers building screenshot-driven GUI agents on local GPUs
Strengths
- Returns grounded actions with box coordinates for Windows, macOS and Android screenshots
- Selectable output formats, e.g. Action-Operation-Sensitive flags sensitive operations
- Includes SFT and LoRA fine-tuning code plus INT4/INT8 inference options
- Weights on HuggingFace, ModelScope, WiseModel and Ascend-targeted Modelers
Weaknesses
- Needs about 29GB VRAM at BF16; INT4 (about 8GB) is discouraged for quality loss
- Not conversational: no continuous dialogue, caller must build the history string
- Output is a strict-format string, not JSON, so callers must parse it themselves
- Online demo cannot control a computer; LoRA fine-tuning needs 70GB on one GPU
What it needs
- GPU required
- Needs PyTorch, transformers
- Models: cogagent-9b-20241220, GLM-4V-9B base, BF16, INT8, INT4
- port 7860
Quick answers
Is cogagent open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does cogagent need a GPU?
Yes, it needs a GPU.
Also in Open models
See all 70| Rank | Project | Score |
|---|---|---|
| 1 | ruviewWiFi CSI sensing on ESP32 for presence, breathing, heart rate and pose | 94 out of 100 |
| 2 | layaSingle-pass multilingual classifier for typed choice, score and yes/no decisions | 88 out of 100 |
| 3 | rf-detrReal-time transformer models for object detection, segmentation and keypoints | 79 out of 100 |
| #4 | timesfmPretrained time-series foundation model for zero-shot forecasting from Google Research | 78 out of 100 |
| #5 | yolov5PyTorch YOLOv5 object detection, segmentation and classification models with training code | 76 out of 100 |