An agent that operates a legacy carrier portal.
Carrier operators still work in legacy portals: a login wall, no API, and forms filled in by hand. This agent does the same work the same way. It reads the rendered screen and operates the controls in a real browser, with no schema underneath.
- Scenarios passed
- 4/4
- Cost of all four runs
- $0.35
- Model
- Claude Sonnet 4.5
- Browser
- Real Chromium
Watch it work
A recorded run of each scenario, one browser frame per step.
- Sign in
- Open the loss form
- Enter the loss
- Confirmation
- Reasoning
- Filling in the operator credentials provided in the task: username "operator" and password "portal123"
- Action
- fill_form(2 fields)
- Outcome
- Filled 2 fields. Typed "operator" into element 0. Typed "portal123" into element 1.
Result: Successfully filed First Notice of Loss for claimant Samantha Davis on policy POL-4480-WC. Peril: water, Date of loss: 2026-05-01, Description: "Pipe burst in the basement causing water damage." Confirmation number: CL-FNOL-100092
23,121 tokens in, 1,260 out, $0.088
How it works
Each turn the agent gets a screenshot and a numbered list of the on-screen controls. It has no source and no schema for the portal, the way a forward-deployed agent meets a customer's legacy system.
It picks one control and one action (click, type, select) and a real browser executes it. Failed inputs (a strict date field, a rejected value) come back as observations it recovers from.
It never fabricates an outcome. An excluded peril, a lapsed or unknown policy, a missing supervisor override: it refers the case to a human instead of forcing the work through.
Scored against the system of record
npm run eval:operator"Run this live" drives the real portal in real time. The function launches a bundled serverless Chromium (the same headless-Chromium-in-a-function pattern these agents deploy on), or connects to a hosted browser when one is configured, and operates the live CarrierLink portal on this site. The recorded runs are there for instant, zero-cost proof; the live run removes any doubt it's real. Live runs are rate-limited (two per visitor every 15 minutes) and budget-capped, and every run is metered on /ops/llm.