Heal your companion
Every Crawler needs a companion inside the box, and yours is the guest agent.
This guide shows you how to check the health of a VM’s guest agent, update it
to the agent in your host’s pomme build, and repair it. Most Pomme commands
that work inside the guest, such as pomme exec, pomme cp, and the security
workflows, need a connected, verified agent. Without it, you’re just shouting
at a window.
For background on what the agent does, see Guest agent.
Before you descend
Section titled “Before you descend”The System requires the following before it unlocks this room:
- Make sure that the VM exists. To list your VMs, run
pomme list. - Build and install the Pomme command-line tool that you want to use. An update installs the agent from the host tool that runs the command, and a repair compares the guest agent with that tool.
Check its vitals
Section titled “Check its vitals”To see the agent’s status, run the following command:
pomme agent status VM_NAMEReplace VM_NAME with the name of your VM.
The command reads the VM’s status without changing the guest. The table output shows the connection state and the agent’s role. For the full set of fields, use JSON output:
pomme agent status VM_NAME --format jsonThe output looks similar to the following:
{ "ok": true, "name": "dev", "guestAgent": { "connection": "connected", "role": "normal", "protocolVersion": 1, "executableDigest": "0000000000000000000000000000000000000000000000000000000000000000", "capabilities": ["agent.describe", "agent.health", "process.start"], "updateState": "current" }, "hostExitCode": 0}The guestAgent object has the following fields:
| Field | Description |
|---|---|
connection |
connected when the host has an authenticated agent session, otherwise disconnected. |
role |
normal for the persistent agent in normal macOS, or recovery for the temporary Recovery session. |
protocolVersion |
The PommeAgentProtocol version that the agent speaks. |
executableDigest |
The SHA-256 digest of the agent executable. |
capabilities |
The operations that the agent advertises. |
updateState |
One of unknown, current, updating, required, failed, or unavailable. |
A stopped VM reports disconnected. A guest that’s still booting connects on
its own; pomme status VM_NAME reports when it does. Patience, Crawler. It’s
putting its boots on.
Update the agent
Section titled “Update the agent”Installing a new pomme command on the host doesn’t change the agent in your
existing VMs. To give a VM the agent from your current host build, with its
new capabilities and log events, update the agent in place. An update doesn’t
boot Recovery or restart the guest.
Before you update, make sure that the following conditions are met:
- Pomme created the VM.
- The VM is running normal macOS, and its agent is connected.
- The VM has no unfinished SIP, AMFI, or MDM operation. To finish one, repeat its original command.
- The VM has no running background jobs or terminal sessions. To check, run
pomme jobs list VM_NAMEandpomme sessions list VM_NAME.
The agent restarts during an update, and the restarted agent doesn’t know
about the background jobs and terminal sessions that the old agent started.
Their programs would keep running, but pomme jobs and pomme sessions could
no longer wait for, attach to, or stop them. For that reason, an update that
finds running jobs or sessions changes nothing and lists the command that stops
each one. Orphaned processes wandering the halls are bad for ratings. Well,
good for ratings. Bad for you.
To update the agent, run the following command:
pomme agent update VM_NAMEReplace VM_NAME with the name of your VM.
Pomme copies the host’s signed pomme executable into the guest, installs it
as the agent, restarts the agent, and waits for it to reconnect with the
SHA-256 digest of the new executable. The output looks similar to the following:
Updated the Pomme agent in dev to 0123456789ab.Copying the executable takes most of the time, usually less than a minute.
The restarted agent runs with the launchd settings that the guest loaded when it started. When an update also changes those settings, they take effect the next time that the guest starts. To apply them right away, restart the VM:
pomme restart VM_NAMEIf the agent already matches the host build, the command makes no change and
reports that the agent is current. JSON output reports "updated": false, and
the command exits with status 0, so you can run it again safely:
pomme agent update VM_NAME --format json{ "ok": true, "name": "dev", "updated": false, "previousExecutableDigest": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", "executableDigest": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", "hostExitCode": 0}After a successful update, Pomme records the new digest for the VM. SIP, AMFI, and MDM workflows then accept the updated agent. For details, see Digest pinning.
If the new agent doesn’t reconnect in time, the command fails and tells you
that the agent is installed but not yet running. Restart the VM with
pomme restart VM_NAME, and then run pomme agent update VM_NAME again to
record the new agent.
Revive the agent
Section titled “Revive the agent”To repair the agent, run the following command:
pomme agent repair VM_NAMEReplace VM_NAME with the name of your VM.
Pomme first checks whether a repair is needed. If provisioning is complete and
the connected normal agent has the required protocol, the required
capabilities, and the same SHA-256 digest as the host command-line tool, Pomme reports that
the agent is already healthy and exits with status 0. It doesn’t change the
VM or its provisioning journal:
Pomme agent is already healthy for VM_NAME.Otherwise, Pomme records the repair in the VM’s provisioning journal before it
starts, installs the agent, verifies it in normal macOS, and then returns the
VM to the state it was in before the command. --final-state accepts only
previous, which is the default.
Pomme boots Recovery only when it has to. Recovery is the long hallway with the keyboard puzzles, and nobody enjoys it except the audience:
- For a VM cloned from a provisioned template that kept its agent credential, Pomme installs the agent in normal macOS, as creation does. For details, see How clones get their agent.
- When an earlier attempt already sent the VM’s credential to the guest, Pomme first stops the VM and boots normal macOS to check for that agent, even when the VM was already running normal macOS. If the agent authenticates with the VM’s credential and reports the pinned digest, Pomme finishes the repair there.
- Otherwise, or when one of these fails, Pomme installs the agent through Recovery.
The repair also records the agent’s verification in the journal, so a later
pomme create VM_NAME --resume only restores the VM’s final state. The output
names the route that the repair took:
OK repaired the Pomme agent in normal macOS for VM_NAME.In JSON output, role is normal for a repair in normal macOS, or
recovery for a repair through Recovery.
If agent provisioning is complete but the agent isn’t healthy for another reason, the repair reports that it has nothing to repair:
Nothing to repair for VM_NAME: agent provisioning is complete. Agent repair does not reconcile SIP or AMFI security transactions.To resume an unfinished SIP or AMFI transaction, repeat its original command instead. For details, see Change SIP and AMFI.
Understand pinned agent artifacts
Section titled “Understand pinned agent artifacts”Each VM’s creation plan pins the exact signed agent executable, by SHA-256
digest, that Pomme installed. When Pomme creates a VM, it keeps a copy of that
executable in an append-only store under
~/Library/Application Support/pomme/AgentArtifacts/sha256, and the local
build script adds every signed build to the same store. When a repair or a
resumed creation reinstalls the agent after you update or rebuild Pomme,
Pomme uses the original pinned artifact, not the new version. If the pinned
artifact is missing or altered, the repair fails and changes nothing. The
System doesn’t accept substitutes. Not even convincing ones.
A repair that reinstalls the agent therefore undoes an earlier
pomme agent update: the VM gets the agent that it was created with, and
Pomme stops accepting the updated digest. To return to your current host build
after such a repair, run pomme agent update VM_NAME again.
Record the replay
Section titled “Record the replay”To keep screenshots of each automatic Recovery navigation step, add --debug:
pomme agent repair VM_NAME --debugPomme prints the private directory and the saved file names to standard error. Delete the directory when you no longer need it, because the screenshots can show local identifiers.