‘We got scared’: Confessions of an ad tech exec’s AI agent experiment

It will probably take a while to translate those ambitions into reality. Along the way, there’s plenty of scope to make costly mistakes and, hopefully, learn from them. In this candid conversation, an ad tech exec talks us through their company’s early efforts to build an agent that could execute media buys — and which did so, in a limited test staged in February — as well as their decision to terminate the pilot when their team realized the potential for misuse.

In our Confessions series, we trade anonymity for candor. Certain parts of the conversation were edited for length and clarity.

What did this AI agent actually do?

It was essentially trading the way a trafficker would, but automated. It looked at insertion orders (IOs) that came in [from brands and media buyers], it set up the campaigns, it set the campaigns live, and it started optimizing them. We were like: “This is amazing!” Well, for about a day.

It spent a few thousand dollars in one go. And then we got scared. What if it accidentally added a zero? What if it makes a mistake overnight or over a weekend? We’d be out of business. So, we ran it for two days, then shut it down. We said [to ourselves], we’re gonna just keep it human, trading on platforms and the DSPs and buying through SSPs, because it felt safer to us at that time. But it works — you can do it.

What LLM was this based on?

We were using OpenAI’s LLM [at the time, GPT-4]. That’s where we built it. This only took us a few days, which was the scarier part. It was not too difficult since we are fairly advanced in AI tooling. And it was using our platform to do the buying. It would get an IO, read it, build the campaign, and set it live. 

Where did the ads run?

Predominantly on CTV, through Freewheel’s SSP. Freewheel is our core partner with regards to premium CTV access; we target a publisher list [it includes networks like ABC, ESPN and Fox], but do not know exactly how many impressions went to each publisher. Once the campaign was set up correctly and running, our platform took over from there.

Were you using your own money to test the ads — placing dummy bids?

No, those were real campaigns from real customers. We did not unleash it on the full company, but it was set up for a handful of client buys. If we’d overpaid, it would have been our money, not theirs.

Did they sign up to that as a pilot scheme?

No, we did that without them knowing. But again, we have humans doing it for them today. There’s a lot of automation tools and bidding [now], so we thought it wouldn’t be any different. 

If you’d brought it to market — this would have been a self-service device?

It would have replaced traffickers — humans setting up campaigns, setting them live, optimizing.

Most of the buys come in at a monthly clip. In the last week of this month [November], most of the buys for December are going to come in. That’s probably 4,000 buys, maybe more, that will all come in like a two-day window. People are taking those buys and putting them in [to our system]; the machine, in theory, would have probably done half of that. 

Did the agent save time?

Honestly, I don’t know if it did it faster than a human. But it wasn’t a human, which saves money and [saves] human error.