peekaboo

Peekaboo is a robust, full-featured macOS UI automation command-line interface designed to bridge the gap between AI agents and desktop application environments. It functions as a powerful utility for programmatic interaction with macOS, enabling agents to see, reason about, and manipulate complex desktop interfaces. By leveraging a snapshot-based architecture and a comprehensive suite of interaction commands, Peekaboo allows for fine-grained control over windows, menus, and system-level components. It is primarily targeted at developers, automation engineers, and AI agent builders who need a reliable, scriptable way to bridge model outputs with real-world application state.

• Full UI visibility through screen capture, window listing, and visual analysis with annotation support.

• Advanced interaction capabilities including precise clicking, drag-and-drop, smart typing, and hotkey sequences.

• Deep system integration for managing application states, dock items, status bar menus, and window bounds.

• Flexible input modes supporting element targeting via IDs, coordinates, or natural language visual queries.

• Native support for JSON output, making it highly compatible with LLM-based agent pipelines and external automation scripts.

• Cross-app workflow automation: execute complex sequences like launching applications, navigating menus, and performing data entry across disparate software environments.

• Requires active Screen Recording and Accessibility permissions to function effectively on macOS.

• Works optimally when combined with agents that can process visual snapshots and map them to CLI command parameters.

• Best practices involve using the peekaboo see command to generate annotated UI maps before executing interaction commands to ensure accuracy.

• Supports various capture engines including modern screen capture kits and classic CG approaches to handle different system requirements.

• Use the --json flag for automated workflows to facilitate machine parsing of application state, window lists, and screen element metadata.

• Recommended to run via polter peekaboo for clean builds and consistent environment state during agent task execution.