phone-agent-skill

The AutoGLM Phone Agent Skill empowers AI agents to perform complex, end-to-end interactions on Android devices. By leveraging the AutoGLM Phone Agent SDK, this skill bridges the gap between natural language intent and physical UI actions. It is designed for developers, QA engineers, and researchers who need to automate repetitive mobile workflows without writing manual test scripts for every UI change. The skill parses high-level user instructions and converts them into precise ADB commands, allowing the agent to navigate through apps, interact with complex layouts, and extract data directly from the device screen.

• Natural Language UI Control: Decomposes abstract goals into granular actions like tapping, swiping, scrolling, and text input.

• Integrated Vision and Interaction: Supports screenshotting and UI text analysis to confirm state changes and ensure accuracy.

• Automated Testing and Data Collection: Ideal for reproducing user journeys, running E2E test suites, and scraping mobile-only data.

• SDK Integration: Utilizes the robust Open-AutoGLM infrastructure for reliable device communication and backend service management.

• Prerequisites: Requires an Android device with developer mode, USB debugging enabled, and a properly configured adb environment.

• Configuration: Ensure the Phone Agent backend service is running and the PHONE_AGENT_ENDPOINT environment variable is correctly set to the service host and port.

• Permissions: The SDK requires accessibility services and overlay permissions on the target Android device to execute touch events and retrieve UI metadata accurately.

• Safety: Execute operations only on test devices or dedicated test accounts. The skill cannot bypass hardware-level biometric authentication (e.g., fingerprint sensors).

• Troubleshooting: If commands fail to execute, verify device connectivity with 'adb devices', check firewall settings for the service port, and ensure the accessibility service is active.