Shippy shows that production agents start with boundaries and audits

Ai2 published an architecture write-up for Shippy, a maritime AI agent for real-time maritime domain awareness in Skylight. The system answers analyst questions over live satellite and vessel signals, not a static dataset.

Shippy treats the agent as a versioned operating system

The authors split the agent into soul, skills and config. The soul is the system prompt with behavioral boundaries. Skills are markdown files with structured frontmatter. Config selects the agent harness, model and runtime. Shippy currently uses OpenClaw and Claude Opus 4.6.

The agent can query the Skylight API for events and vessel data, look up EEZ and MPA boundaries, interpret vessel track data and generate map links. Crucially, it is not allowed to make legal determinations about whether a vessel is breaking the law or speculate beyond the data.

Deterministic tools narrow the space for agent mistakes

Ai2 says early prototypes let the agent construct API calls directly. That created bugs in pagination, geometry encoding and filters. Shippy now talks to Skylight through a purpose-built CLI with typed flags, help text, error messages and JSON file output.

That is the practical enterprise lesson: the riskier the domain, the less freedom the model should have at the interface. The agent can plan and explain, but the tools need to be predictable.

Isolation is a safety feature, not just infrastructure

Skylight serves hundreds of government agencies and NGOs across more than 70 countries. Each Shippy user gets a temporary isolated session provisioned by Mothership as a dedicated Kubernetes environment. The user's JWT scopes API calls to their own data.

That is the right level of paranoia. For an agent that can run code, write files and touch sensitive operational data, a shared session is an incident invitation.

Evals have to measure the whole agent, not one model answer

Ai2 evaluates Shippy with scenarios from subject-matter experts, weighted rubrics and an LLM judge scoring criteria from 0 to 1. The target is the whole system: model, skills, sandbox and live data access.

The next signal is whether these evals catch regressions when the model, skill or CLI changes. Production agents do not fail on a leaderboard. They fail when a wrong query sends a human to the wrong place.

Lilith's verdict

Shippy is a reminder that a serious agent is not a clever head in a jar. It is a guard on the dock that must know where it can step, who owns the map and when to call a human.