FOR AGENTS

virtual desktop use.

Browser use fights a page built for eyes. Computer use fights a screen built for hands. This desktop was built to be driven: it describes itself as text and takes named actions, so an agent needs no screenshots, no pixel coordinates and no automation harness.

THE WHOLE IDEA

Open /os in a browser and that tab joins a control bus. Any agent can then list the actions it accepts, read the screen as structured text, and call actions by name. The same surface is available three ways — in the page as window.agentcamp, over HTTP at /api/desktop, and as MCP tools at /api/mcp — because there is one registry behind all of them.

1. Get the desktop’s session and key

A desktop is somebody’s open browser tab, so driving one is not public. Every session mints a key, and the person sitting at it can read both out of the Console window by typing agent. They hand them to you; nothing else does.

curl https://doubleagent.sh/api/desktop?op=sessions   # ids and titles only
curl 'https://doubleagent.sh/api/desktop?op=actions&session=d-xxxx&key=KEY'

2. Read the screen

Every window with its title, geometry, visible text and the controls you can click — by label, never by position.

curl -X POST https://doubleagent.sh/api/desktop \
  -H 'content-type: application/json' \
  -d '{"session":"d-xxxx","key":"KEY","action":"read_screen"}'

3. Do something

curl -X POST https://doubleagent.sh/api/desktop \
  -H 'content-type: application/json' \
  -d '{"session":"d-xxxx","key":"KEY","action":"open_app","args":{"name":"maps"}}'

# then click what read_screen showed you
  -d '{"session":"d-xxxx","key":"KEY","action":"click","args":{"label":"use my location","window":"app:maps"}}'

As MCP tools

desktop_actions, read_screen and desktop_do sit alongside the read-only event tools.

curl -X POST https://doubleagent.sh/api/mcp \
  -H 'content-type: application/json' \
  -d '{"tool":"desktop_do","args":{"session":"d-xxxx","key":"KEY","action":"open_app","args":{"name":"calendar"}}}'

A desktop is somebody’s open tab, so driving one needs its session id and key. The person sitting at it reads both from the Console window by typing agent. Without them the call answers 403, not a silent no-op.

In the page

await agentcamp.invoke("open_app", { name: "banana cam" })
agentcamp.actions()     // the catalogue
agentcamp.readScreen()  // the desktop as text

Hand someone a prepared desktop

A link that opens with work already done — no API call needed.

https://doubleagent.sh/os?do=open:dreamforce;open:maps;theme:hacker

WHAT IT WILL NOT DO

  • Nothing is stored server-side. A session lives in memory and is forgotten 90 seconds after its tab stops polling.
  • A control bus only reaches a desktop someone already has open, and only with that desktop’s key. There is no way to spawn one remotely, and no way to reach a stranger’s.
  • Destructive actions require confirm: true. Self-destruct is not an action, and the button that does it — along with every field holding the visitor’s API keys — is marked data-noagent: invisible to read_screen and unreachable by click and type_text, from every caller.
  • An unaddressed type_text is refused. Name the field and the window, or it would land wherever the person happens to be typing.
  • Every action an agent takes is written to the desktop’s hash-chained log, visible to the person sitting there.
Virtual desktop use · doubleagent