What building an agentic AI system to run my own pipeline taught me
I built the thing everyone's pitching before I started pitching it — here's what actually held up.
I built it because I didn't trust the pitch
Every vendor selling "agentic AI for sales" was selling a demo. Clean data, one lead source, a happy path from trigger to reply. I help run a technical staffing firm — I needed something that would touch real, messy pipeline: inconsistent contact data, candidates who ghost, clients who change scope mid-search. So I built an agentic system to run TechNet's own outbound. Not a pilot. Production. Real pipeline, real risk.
That distinction matters more than anything else in this piece. A system that only has to survive a demo will always look better than one that has to survive Tuesday.
What "agentic" actually buys you
The honest version: an agent is very good at the parts of the job that are research and judgment under uncertainty, done thousands of times, where a human would get bored and start skipping steps long before an AI would. Pulling context on a company, drafting a message that reflects that context, deciding whether this contact fits this campaign, routing exceptions to a human — that's real work an agent does faster and more consistently than a person grinding through row 400 of a spreadsheet.
What it does not buy you is judgment about what should happen next in a relationship. It doesn't know that a prospect went quiet because they're mid-reorg, not disinterested. It doesn't know which technical objection is a real blocker versus a stalling tactic. Pattern-matching on text gets you close. Close isn't good enough when the thing you're sending has your name on it.
Where it breaks in practice
Three places, consistently:
- Ambiguous signals. An agent reasons well over clean information and reasons confidently over dirty information — which is the dangerous case, because confident and wrong looks identical to confident and right until a human checks it.
- Edge cases that aren't edge cases. The "unusual" scenario — a contact who changed roles, a company that got acquired mid-sequence — shows up constantly once you're running at volume. Systems built around the happy path treat this as noise. Production systems have to treat it as the actual job.
- Silent drift. An agent doesn't get worse in a way that trips an alarm. It gets worse in a way that looks fine in the logs and only shows up three weeks later as a soft decline in reply quality or fit. You need instrumentation built for that from day one, not bolted on after you notice the drift.
None of this means agentic systems don't work. It means they work inside a boundary, and the boundary has to be designed on purpose, not discovered after something goes out that shouldn't have.
Why the approval boundary isn't a limitation — it's the design
The instinct when you build something that works is to remove the human and let it run. I went the other way. Every send in our system routes through a human approval step. Not because the agent can't draft a good message — it usually can — but because "usually" is not a word you want attached to anything with your company's name in the From field.
The approval step also turns out to be where the real engineering value shows up. It's the point where you find out what the agent is actually doing wrong, at a cadence fast enough to fix it before it compounds. A fully autonomous system hides its failure modes until they're expensive. A human-in-the-loop system surfaces them daily, cheaply, as a normal part of operating it. That's not a safety feature bolted onto the architecture. It's an architecture decision that happens to also be the safe one.
The gap between demo and production
A demo has to impress someone once. A production system has to be right, or safely wrong, thousands of times in a row, across data it wasn't shown in advance, under a workload that doesn't stop for the parts that are hard. That gap is where most "agentic AI" projects die — not in the model, in the surrounding system: the validation, the exception handling, the audit trail, the boring plumbing nobody demos.
If you're technical and evaluating whether to build this in-house or bring in help, that's the actual question worth asking a vendor: not "can your agent draft a message," almost everything can, but "what happens when it's wrong, and how do you know before your customer does." If they don't have a specific answer, they haven't run one in production.
Building or fixing an agentic system of your own?
I've built one that runs real pipeline with a human approving every send. Happy to talk through what's transferable to yours — architecture, failure modes, where the boundary should sit.
Tell us the seat