# Driving the iPhone

> The operating loop and calibrated rules that make an agent reliable on a real iPhone - targeting, scrolling, typing, recovery and safety.

Source: https://zeptap.com/docs/agents/driving



A real iPhone differs from a browser: there is no DOM, taps land on pixels, and animations take time. These rules come from eval runs on a real iPhone, and extend the instructions the server sends in its `initialize` response.

## The loop [#the-loop]

1. **Look.** Start with `screenshot`. Read the screen before acting on it.
2. **Act.** Make one action.
3. **Verify.** Read the screenshot the action returns. It is taken after the screen settles, so it shows the result. Continue only when it shows what you expected.

When a result note says `screen still changing`, call `wait` before the next action: an animation or load is still running, and a tap now could land on a moving target.

## Targeting [#targeting]

* **Prefer text.** For any visible label, use `tap_text` (or `find_text` for coordinates). OCR finds the label's exact center; estimating from the image is the most common cause of missed taps.
* **Tap centers.** When tapping by coordinates, aim at the middle of the control.
* **Pick among duplicates** with `tap_text`'s `index` (0 is the topmost match), or narrow with `find_text` first.
* **Icons without text** (toolbar glyphs, toggles): tap by coordinates from the latest screenshot.
* **Ignore two overlays.** The small dark dot is the pointer. The large translucent circle is the AssistiveTouch button. Neither is part of the app. When the AssistiveTouch button covers a control, scroll the content out from under it.
* **The status bar always reads 9:41** in captures. It is not the real time, and battery and signal are not real either.

## Scrolling [#scrolling]

| Need                           | Use                                       | Calibration                                                   |
| ------------------------------ | ----------------------------------------- | ------------------------------------------------------------- |
| Move a list a precise distance | `drag` at the default 800 ms              | Content moves about 0.85× the drag distance, with no momentum |
| Reveal content further down    | `drag` from a lower point to a higher one | For example `from_y: 1000` → `to_y: 400`                      |
| Jump far through a long list   | `flick` `up`                              | One flick can cover a whole Settings list                     |
| Turn a picker wheel            | `drag`                                    | About 50 px per step                                          |
| Move a slider                  | `drag` along the track                    |                                                               |

Start scroll gestures in the content area, away from the screen edges: an edge swipe from the left goes back, and one from the bottom goes home.

After each scroll, re-find the target (`find_text` or `tap_text`) rather than reusing coordinates from before the scroll.

## Typing [#typing]

* **Focus first.** Tap the field, then confirm the keyboard or cursor is showing before `type_text`.
* **US-keyboard ASCII only.** Any other character (accents, emoji, curly quotes, `₹`) rejects the whole call with nothing typed. Rephrase the text in ASCII.
* **iOS edits what you type.** Autocorrect, auto-capitalization and smart quotes can change it. Read the returned screenshot, and fix mistakes with `press_key` `backspace` (use `repeat`) or select-all with `press_key` `a` + `command`.
* **Submit** with `press_key` `enter`, or tap the on-screen button.
* **Long text** types at about 80 ms per character.

## Navigating and recovering [#navigating-and-recovering]

| Situation                                          | Do                                            |
| -------------------------------------------------- | --------------------------------------------- |
| Start an app                                       | `open_app` with the name shown under its icon |
| Leave an app                                       | `home`                                        |
| One screen back                                    | `back`, or tap the app's back button          |
| Spotlight, a menu, a sheet or the keyboard is open | `dismiss`                                     |
| Control Center or Notification Center is open      | `home`                                        |
| Lock screen, passcode pad, or no screen image      | `unlock`, once. If it fails, tell the user    |
| Unsure where you are                               | `home`, then start again from the Home Screen |

Settings, Safari and other system apps restore their last screen when reopened. After `open_app`, read the screenshot before assuming you are on the app's first screen.

## Safety [#safety]

Confirm with the user before any action that is hard to undo or reaches other people: sending a message or email, posting, buying, deleting, changing an account or a privacy setting, or accepting a permission prompt. The phone is real and the actions are real.

Leave the phone as you found it: undo test data you created, and restore settings you changed.