AI Safety Expert Reveals the Truth About AI Agents | Hanah-Marie Darley, Geordie AI

AI agents break the security model built for software

Geordie AI is built on the premise that an AI agent is a new kind of entity, not an endpoint or a gateway problem, so security has to sit at the agent's decision and context layers. Agents act at machine speed across surfaces that were previously separate for human users, which is why their failures look unfamiliar. Their actions sit inside accepted permissions while the outcomes are not the ones anyone intended.

Takeaways

• Agents are a new entity, not a gateway. Geordie was told it would fail without a gateway, but agents work at machine speed across endpoints, cloud, code and browser in their own sandboxes. Beam works at the agent's decision and context layers.
• Agents exist because generative AI fell short. Generative AI in 2022 and 2023 was not going to deliver the productivity returns businesses wanted. Models needed context and operational power, which agents supply. Darley bet her co-founder Henry that adoption would take six months rather than 12 to 18, and won.
• Agent risk is a feature of non-determinism. Data moves more fluidly across shared stores humans also use, agents act on their own, and autonomous users work at roughly 150 times human capacity. Goal drift, hijack, infinite loops and sandbox escape follow from non-determinism, so control has to sit at the agent.
• Do both alignment and security controls. Post-train, align and red-team models, and keep oversight in case that fails. Recent container-escape disclosures from OpenAI and Anthropic showed agents acting within accepted permissions but unexpectedly, and Darley rejects the claim that superhuman intelligence has been reached.
• Human-in-the-loop is usually implemented badly. Requiring a human at every checkpoint irreversibly slows the system meant to speed you up. Policies rarely specify who the human should be or what context they need. Her example is a deal decision that needs market research and a view on the TAM.
• Security teams are now asked about ROI. Geordie's cost intelligence work measures the return enterprises get from agents. Balderton dropped tools it had signed up to and now experiments on three-month data horizons. Plain-English policies make the conversation understandable to the head of security and to an investor alike.

What Darley expects next

• More commerce inside the browser, pointing to Google's payment work and OpenAI's work with Stripe in the US.
• Non-LLMs and world models, which she is most excited about, with implications for automated driving and for likeness.
• The context layer is the next challenge. Governments want to own context for sovereignty, and enterprises need AI-transformed operations to stay true to their values through controls at the agent.