#6666th of 102 in Agent platforms
esp32-iphone-agent
AI agent that taps, swipes and types on an iPhone via an ESP32 board
Live demo ↗
(opens in a new tab)Documentation ↗
(opens in a new tab)Website ↗
(opens in a new tab)Repository on GitHub ↗
(opens in a new tab)
- Stars
- 4.1k
- License
- MIT
- Last commit
- Oct 2026
- Last release
- Oct 2026
- Language
- C
2,142 on 17 Oct 20264,096 on 14 Nov 2026
From GitHub's daily star counts.
Missed the latest GitHub check; these facts may be older.
Overview
An ESP32-S3 board plugged into an iPhone receives the screen over AirPlay and acts as a USB HID mouse and keyboard. A SwiftUI app on the same phone sends screenshots to any OpenAI-compatible vision model and runs planner and actor steps through AssistiveTouch. It also ships an MCP server (npx esp32agent mcp) so coding agents can drive the phone.
Who it is for: Tinkerers automating iPhone tasks with vision models and cheap hardware
Strengths
- Needs no computer at run time: only the phone, a ~$7 board and a model endpoint
- Works with any OpenAI-compatible vision endpoint, including vLLM, llama.cpp and Ollama
- Every run is stored on the phone with step-by-step replay and trace export
- Includes an MCP server and Claude Code plugin for driving the iPhone from coding agents
Weaknesses
- Requires specific hardware: an ESP32-S3 with native USB and PSRAM
- Mirrors at 480x1040, up to 5 fps; each step takes a few seconds
- Cannot see passcode, Apple Pay or DRM screens; landscape apps unsupported
- Phone Wi-Fi is taken by the board hotspot, so the model is reached over cellular
What it needs
- no GPU
- Needs ESP32-S3 board (Seeed XIAO ESP32S3, 8 MB PSRAM), iPhone with iOS 18+ and cellular data, OpenAI-compatible vision model endpoint, arduino-cli and Xcode to build from source
- Models: OpenAI-compatible vision APIs, vLLM, SGLang, llama.cpp, Ollama
Also in Agent platforms
See all 102| Rank | Project | Score |
|---|---|---|
| 1 | AutoGPTRole-based AI agents coordinated by one lead agent, plus a visual workflow builder | 95 out of 100 |
| 2 | daily_stock_analysisLLM-driven daily analysis of watchlist stocks across six markets, pushed to chat apps | 95 out of 100 |
| 3 | LangflowVisual builder for AI agents and workflows, served as API or MCP server | 94 out of 100 |
| #4 | deer-flowAgent harness running sub-agents, memory and sandboxes with pluggable skills | 93 out of 100 |
| #5 | DifyVisual builder for LLM workflows, RAG pipelines and agents, with APIs | 92 out of 100 |