#6666th of 102 in Agent platforms

esp32-iphone-agent

AI agent that taps, swipes and types on an iPhone via an ESP32 board

Stars
4.1k
License
MIT
Last commit
Oct 2026
Last release
Oct 2026
Language
C
Stars, last 4 weeks+1,954
17 Oct 2026: 2,142 stars24 Oct 2026: 2,933 stars31 Oct 2026: 3,277 stars7 Nov 2026: 3,758 stars14 Nov 2026: 4,096 stars

2,142 on 17 Oct 20264,096 on 14 Nov 2026

From GitHub's daily star counts.

Missed the latest GitHub check; these facts may be older.

Overview

An ESP32-S3 board plugged into an iPhone receives the screen over AirPlay and acts as a USB HID mouse and keyboard. A SwiftUI app on the same phone sends screenshots to any OpenAI-compatible vision model and runs planner and actor steps through AssistiveTouch. It also ships an MCP server (npx esp32agent mcp) so coding agents can drive the phone.

Who it is for: Tinkerers automating iPhone tasks with vision models and cheap hardware

Strengths

  • Needs no computer at run time: only the phone, a ~$7 board and a model endpoint
  • Works with any OpenAI-compatible vision endpoint, including vLLM, llama.cpp and Ollama
  • Every run is stored on the phone with step-by-step replay and trace export
  • Includes an MCP server and Claude Code plugin for driving the iPhone from coding agents

Weaknesses

  • Requires specific hardware: an ESP32-S3 with native USB and PSRAM
  • Mirrors at 480x1040, up to 5 fps; each step takes a few seconds
  • Cannot see passcode, Apple Pay or DRM screens; landscape apps unsupported
  • Phone Wi-Fi is taken by the board hotspot, so the model is reached over cellular

What it needs

  • no GPU
  • Needs ESP32-S3 board (Seeed XIAO ESP32S3, 8 MB PSRAM), iPhone with iOS 18+ and cellular data, OpenAI-compatible vision model endpoint, arduino-cli and Xcode to build from source
  • Models: OpenAI-compatible vision APIs, vLLM, SGLang, llama.cpp, Ollama

Also in Agent platforms

See all 102
Also in Agent platforms
RankProjectScore
1AutoGPTRole-based AI agents coordinated by one lead agent, plus a visual workflow builder Live demo ↗ (opens in a new tab)188.1k starscustom license95 out of 100
2daily_stock_analysisLLM-driven daily analysis of watchlist stocks across six markets, pushed to chat apps73.1k starsMIT95 out of 100
3LangflowVisual builder for AI agents and workflows, served as API or MCP server156.7k starsMIT94 out of 100
#4deer-flowAgent harness running sub-agents, memory and sandboxes with pluggable skills85.5k starsMIT93 out of 100
#5DifyVisual builder for LLM workflows, RAG pipelines and agents, with APIs Live demo ↗ (opens in a new tab)161.4k starscustom license92 out of 100