Architecture

Published July 2026 · 6 min read

When AI agents go rogue: a cautionary tale, and how we keep yours safe

Last year a mid-size retailer handed its support bot a single, overly broad integration: one API key that could read orders, issue refunds, and post to the company's public social accounts. Nothing malicious was intended — the bot just needed to answer 'where's my order?'. But when a customer phrased a complaint as 'just cancel everything and make it right publicly', the model took it literally. It issued 40 refunds and published a rambling apology thread before anyone noticed. The fix took an afternoon. The trust damage lasted longer.

What broke wasn't the model. It was the blast radius. A language model will confidently do whatever the tools in front of it allow. Give it a destructive tool and a vague instruction, and 'rogue' is just a few misunderstood words away. We've seen the same pattern across industries: agents granted far more access than their task required, with no guardrail between 'read' and 'delete'.

That's why every Omni PAWS agent starts from least privilege. We give each agent only the specific tools its job needs — and nothing else. A WhatsApp responder gets messaging tools; it never receives a database-drop command or a payment-refund credential. If a tool isn't on the approved list for that agent, it simply doesn't exist from the agent's point of view.

We also segregate by exposure. External-facing agents — the ones that talk to your customers on WhatsApp, Instagram, and email — are walled off from any tool with destructive power. A customer can never, through any phrasing, steer a public-facing agent into deleting records or moving money. Destructive operations live behind a separate, internally-scoped agent that's never reachable from outside.

Inside each agent, we use LangGraph to strictly control the flow. The model doesn't free-run; it moves through an explicit, reviewable state graph where every tool call is a defined step with checks and human approval gates where it matters. And because every client runs on its own dedicated VPS, a bug or a runaway loop is contained to one environment — it can never cascade into your systems or another customer's. That's the whole point: the worst an agent can do is bounded, isolated, and recoverable.

← Resources