Ghost hands (display)
Time to haunt a screen, Crawler. This guide shows you how to drive a VM’s
display from the host: press keys, type text, click at coordinates, and
capture screenshots. The pomme ui commands deliver input directly through
the VM’s virtual keyboard and pointer and read its framebuffer, so they work in
normal macOS and in Recovery without a guest agent.
These commands don’t open a host window, move the host pointer, or change the frontmost app on the host. Your own desk stays untouched. Only the box gets haunted.
Each command that acts on a VM names it with --vm VM_NAME. If you set
POMME_VM_NAME, you can omit --vm.
Before you descend
Section titled “Before you descend”The System requires the following before it hands over the ghost hands:
- Start the VM. For example, to start it in Recovery, run
pomme start VM_NAME --mode recovery. - Don’t send manual input while Pomme is provisioning the VM automatically,
for example during
pomme createor a SIP change. Automatic provisioning owns an exclusive lease on the VM’s display, and manual input can disrupt its navigation. - Run one
pomme uicommand at a time for each VM. If another display operation is in progress, the command fails withA direct VM UI operation is already in progress.
Two ghosts, one keyboard, zero good outcomes. The viewers would love it. Don’t.
Capture a screenshot
Section titled “Capture a screenshot”To see what’s on the guest display, save a screenshot to a PNG file on the host:
pomme ui screenshot --vm VM_NAME --output OUTPUT_PATHReplace the following:
VM_NAME: the name of your VM.OUTPUT_PATH: the host file to write, for example/private/tmp/pomme-lab/screen.png. The parent directory must already exist, and the path can’t be a directory or a symbolic link.
Press a key
Section titled “Press a key”To press one key or key combination, run pomme ui key and name the key with
--key:
pomme ui key --vm VM_NAME --key returnpomme ui key --vm VM_NAME --key cmd-shift-tpomme ui key --vm VM_NAME --key ctrl-f2Replace VM_NAME with the name of your VM.
Modifier prefixes chain from left to right, and you can use + in place of
-. For example, cmd+shift+t is the same as cmd-shift-t. To list the key
names, modifiers, and aliases that Pomme accepts, run pomme ui keys, or see
Key names.
Pomme doesn’t accept numeric HID scan codes. Use the names, Crawler. The System isn’t decoding hex for you.
Press a sequence of keys
Section titled “Press a sequence of keys”To press several keys in order, run pomme ui key-sequence and put the keys
after --:
pomme ui key-sequence --vm VM_NAME -- down down returnReplace VM_NAME with the name of your VM.
Pomme reads every value after -- as a key, so a key can’t be mistaken for a
VM name or a flag.
Type text
Section titled “Type text”To type text into the focused field, run pomme ui type. Supply the text in
exactly one of the following ways:
-
With
--text:Terminal window pomme ui type --vm VM_NAME --text '/usr/bin/id -u' -
With
--text-env, which reads the text from a host environment variable:Terminal window pomme ui type --vm VM_NAME --text-env VARIABLE_NAME
Replace the following:
VM_NAME: the name of your VM.VARIABLE_NAME: the name of an environment variable that contains the text.
Pomme types any single character on a US keyboard. An uppercase letter or a shifted symbol implies Shift.
To replace the contents of the focused field instead of adding to them, add
--replace. Pomme presses Command-A before it types.
Click a point on the display
Section titled “Click a point on the display”To click a point on the guest display, give its coordinates in points, measured from the top-left corner:
pomme ui click --vm VM_NAME --x 640 --y 400Replace VM_NAME with the name of your VM. Coordinates must be zero or
greater. Pomme VMs use a 1280 × 800 display.
To find coordinates, capture a screenshot first and measure the element’s position. Clicking blind is how Crawlers end up pressing the wrong button on live television.
Set a time limit
Section titled “Set a time limit”Each pomme ui command that talks to a VM accepts --timeout in seconds. For
example:
pomme ui screenshot --vm VM_NAME --output OUTPUT_PATH --timeout 30