Agents That Don't Hallucinate Your Money Away: Guardrails for Automated Media Buying
I run an 11-agent DSPy orchestration layer against live ad spend. Real money, real traffic sources, real consequences if something goes wrong at 3am while I’m asleep. This piece is about the guardrails that make that survivable, not the AI. The AI is the easy part. The guardrails are the actual engineering.
Autonomy is a dial, not a switch
The framing I see most often is binary: either the agent is “autonomous” and does things, or it’s “human-in-the-loop” and asks permission for everything, which people quietly treat as the training-wheels version you graduate out of. I think that framing is wrong for anything touching money, and building against live spend forced me to replace it with a different model.
Autonomy isn’t one setting. It’s a dial per action type. My system reads freely — pull stats, check campaign health, analyze traffic patterns, no gate, no approval, go as fast as the API lets you. Every action that affects spend goes through a write-approval gate. That’s not a temporary scaffold I’ll remove once I trust the system more. It’s the permanent shape of the architecture, because the two action classes have permanently different risk profiles. Reads are cheap to be wrong about — you re-run the query. Writes that move money are not cheap to be wrong about, and “the model was 95% confident” is not a number that comforts anyone when the wrong number gets 95% confident about the wrong campaign.
What’s actually gated, mechanically
The write-approval gate isn’t a suggestion box. Every spend-affecting action — pausing a campaign, adjusting a bid, reallocating budget, killing a traffic source — stops and waits for a human sign-off before it executes. It has a 900-second approval timeout. If nobody answers in 15 minutes, the action doesn’t happen. That timeout is deliberate: silence is not consent. A system that defaults to “proceed if unanswered” is a system that will eventually spend money because someone was in the shower.
Approvals come through Telegram, and the callback buttons are HMAC-signed. That detail matters more than it sounds like it should. An approval flow that isn’t cryptographically verified is a system where anyone who can see or guess the callback structure can approve their own forged spend action. If the whole point of the gate is “a human authorized this,” the authorization channel has to be unforgeable, or the gate is theater.
Underneath that sits a circuit breaker module and per-traffic-source rate limiters. The circuit breaker exists for the failure mode where something upstream is degrading in a way that isn’t a clean error — latency creeping up, success rates dropping — and the system needs to stop hammering a sick dependency instead of retrying its way into a worse outage. The rate limiters exist because different traffic sources have different tolerances and different costs of being hit too hard, and a single global rate limit treats a source that punishes you for bursts the same as one that doesn’t, which is just wrong.
Data classification and budget caps
Not every guardrail is about spend directly. There’s a data-classification layer that guards what gets pulled into prompts and what doesn’t — because an agent with broad read access to your operational data will happily include things in a reasoning chain that shouldn’t be there, not out of malice, just because nothing told it not to.
And there’s an LLM call budget cap: 150 calls per hour. This one surprised people when I’ve described it before, because it sounds like a cost-control measure, and it is one, but that’s not the main reason it’s there. The main reason is that a runaway agent loop — one stuck re-reasoning about the same decision, retrying a call that will never succeed, or caught in some recursive delegation pattern — needs a hard ceiling that isn’t “I’ll notice eventually.” 150/hour is a circuit breaker for the orchestration layer itself, not just for the wallet.
OPERATIONAL_MODE=crisis
There’s a switch — OPERATIONAL_MODE=crisis — that changes system behavior when things are actively going wrong. I’m not going to pretend this is some elegant state machine; it’s an environment variable that tightens the defaults. The point isn’t the implementation, it’s the acknowledgment baked into having it at all: normal operating assumptions don’t hold during an incident, and pretending your guardrails should behave identically in calm and crisis is how you end up either too slow to react or too loose right when tightness matters most.
The human isn’t a bottleneck. The human is the product feature.
I want to push back on the framing I hear constantly in this space, which is that human approval steps are friction to be engineered away — a v1 compromise you eliminate as trust in the model grows. I don’t think that’s right, and building this system against real money changed my mind concretely, not just philosophically.
The approval gate isn’t there because the model isn’t good enough yet. It’s there because “an action that moves real money happened because a human looked at it and said yes” is not a bug to be optimized out — it’s the actual product. The value I’m selling myself, running this system, is not “spend gets managed with zero human touch.” It’s “spend gets managed fast, with the parts that don’t need judgment automated completely, and the parts that do routed to a human in under 15 minutes with full context and a forge-proof approve button.” That second thing is worth more than the first thing, not less.
Autonomy that can’t be interrupted isn’t autonomy, it’s abdication. The dial matters more than the switch. Read fast, write carefully, and build the approval step like it’s core infrastructure — because it is.