A helper follows a written step list on an iPhone, reads the screen between taps and stops if the screen is not the one expected.
Browser & Desktop Automation
Cross-sector
Agent skill
Productised
Local/private
2026
The problem
Some jobs exist only inside a phone app, with no desktop version and no export. Someone had to tap through them by hand. A wrong screen was easy to miss, and the next tap went ahead anyway.
What it does
The helper is pointed at the phone and given a written list of taps. It reads the screen between steps and stops with a report if the screen does not match the step. So far this has been built and tested, and has not been run as a finished job on a real handset.
Outcomes
A phone-only task can be repeated from a written list.
A mismatch stops the run before the next tap.
The stop report shows what was on the screen.
Simplified mockup
Interactive preview with sample data. Nothing is sent to a server.
Handover on the handset
Handover
Ready
3 collections are waiting. The agent has not tapped yet.
4418 Jordan Hale · UK 2 · Bay 2
4420 Nia Cole · UK 13 · Bay 2
4426 Sam Reed · UK 1 · Bay 4
1. Open collections
Expect: Collections, 3 waiting
2. Open order 4418
Expect: Order 4418, Jordan Hale, UK 2
3. Confirm handover
Expect: Collected, receipt sent
List loaded, not started
Stop report
Stopped before step 3. Step 2 expected order 4418, Jordan Hale. The screen read: Sign in again. This handset session has ended.
Expected screen for each step
Step 1 · Open collections
Title must be Collections. The body lists three waiting orders, including 4418.
Step 2 · Open order 4418
Title must be Order 4418. The name on screen must be Jordan Hale, size UK 2, bay 2. A sign-in wall is a mismatch.
Step 3 · Confirm handover
Title must be Collected. The receipt line names Jordan Hale. This step is not tapped if step 2 failed.
Earlier stop reports
When
Stopped
Screen reading
Next tap
Yesterday 16:40
Step 2
No network. Order 4402 did not open.
Not tapped
Tuesday 11:05
Step 1
Wrong shop. Bay list was for the other site.
Not tapped
Monday 09:20
Finished
Collected. Receipt sent for order 4388.
All three
Each row is a canned reading from a previous list. The live run is on the Run tab.
How it works
Load the step list and the expected screen for each step.
Read the phone screen before the first tap.
Perform the tap or the key entry.
Read the screen again and compare it with the expected step.
Stop and show what was read if the screen does not match.
Pipeline
flowchart LR
A[Steps] --> B[ReadScreen]
B --> C[Tap]
C --> D[Check]
D --> E[Next]
D --> F[StopReport]