#3333rd of 118 in Agent frameworks and SDKs
agent-desktop
Desktop app control for AI agents via the OS accessibility tree
- Stars
- 21.9k
- License
- Apache-2.0
- Last commit
- Nov 2026
- Last release
- Nov 2026
- Language
- Rust
1,096 on 20 Apr 202621,863 on 9 Nov 2026
From GitHub's daily star counts.
Checked on GitHub.
Vercel is hiring: 87 open roles
Overview
agent-desktop is a Rust CLI and C library that reads a desktop app's accessibility tree (AX API on macOS, UI Automation on Windows, AT-SPI on Linux) and returns it as JSON with a ref per actionable element. The calling agent clicks or types on refs and re-reads the UI to verify; no model is bundled. It also serves its commands as MCP tools over stdio.
Who it is for: Developers giving their own agents control of desktop apps
Strengths
- Uses accessibility tree refs instead of screenshots or pixel matching
- Runs on macOS 13+, Windows 10 1809+, and Ubuntu 24.04+
- Ref actions are headless by default, without moving the pointer or taking focus
- Ships a CLI, a C library, and an MCP server (agent-desktop mcp)
Weaknesses
- No model or agent loop included; you must supply and drive the agent
- Refs go stale when the UI changes, so snapshots must be retaken
- Four commands (key-down, key-up, mouse-down, mouse-up) refuse to run without a daemon
- Linux support is documented only for Ubuntu 24.04 or later
What it needs
- no GPU
- Needs OS accessibility permissions
What changed
From the release notes. No flag means the note does not say so, not that the upgrade is safe.
Quick answers
Is agent-desktop open source?
Yes. Its code is on GitHub under the Apache-2.0 license.
Does agent-desktop need a GPU?
No, it runs without one.
Also in Agent frameworks and SDKs
See all 118| Rank | Project | Score |
|---|---|---|
| 1 | ppt-masterAgent skill that turns documents into natively editable PowerPoint decks | 95 out of 100 |
| 2 | openai-agents-pythonPython SDK for multi-agent workflows with handoffs, guardrails and tracing | 91 out of 100 |
| 3 | mastraTypeScript framework for building AI agents, workflows and MCP servers | 91 out of 100 |
| #4 | browser-usePython library that lets an LLM agent control a web browser | 90 out of 100 |
| #5 | copilotkitSDK for adding agent chat, generative UI and shared state to apps | 90 out of 100 |