WTF are tokens?

Every prompt, summary, workflow automation or agency consumes tokens, which makes the new variable cost of knowledge work — a meter quietly running beneath everything from marketing reports to media plans.

That meter is starting to matter more.

What is a token in today’s currency?

When someone sends a prompt to a large language model or receives a response, the system breaks language into small segments — pieces of works, full words, punctuation or even spaces. These fragments are called tokens. During training, the model learns how tokens fit together and which combinations tend to appear in sequence. When it generates a response, it predicts the next token, then the next, over and over. 

That’s the technical explanation. The commercial one is simpler: tokens are how Ai usage gets metered and billed. Both the text you send in and the next the model generates are counted. The more tokens processed, the more compute is used — and the higher the cost.

As Andrew Frank, a Gartner vip and analyst who focuses on the marketing industry, put it: “In the new model, compute is going to become a core cost, meaning it has to be forecast, managed and absorbed just like labor in an agency P&L.”

Why do tokens suddenly matter to agencies? 

Because they’re using them more — and more consistently.

Early AI use in agencies was sporadic — a prompt here, a deck summary there. Now AI is moving into always-on workflows: planning, reporting, segmentation, optimization and creative iteration. That changes usage. Instead of occasional prompts, agencies are running systems that monitor performance, forecast outcomes and assist decisions continuously. Each of those actions consumes tokens. And because they often run persistently rather than episodically, token usage starts to look less like occasional software activity and more like ongoing infrastructure load.

What does it actually mean? 

Slowly, marketing companies are moving from selling time to managing compute, and that reshapes how they make money. Take agencies. In the traditional agency model, costs scaled mostly with labor. Tools were overheads, buried inside those rates. Tokens disrupt that logic. The AI-driven work they power now carries a variable infrastructure cost underneath it. The more token-based inference cost it absorbs.

Are tokens repricing the ad market?

At a market level, tokens are still mostly burned in experimentation — creative automation pilots, AI-powered search features, internal research copilots and customer service bots. These are edge use cases where companies test what the technology can do.

But when those projects prove their return, they don’t stay experiments. They get wired into everyday workflow. That’s the turning point — when token usage shifts from marginal to structural. And it’s already underway at companies building AI into their products and operations, not just using it as a tool. Ad tech vendors layer in AI optimization, agencies building production engines and platforms shipping AI-native features are already managing token cost as part of their unit economics.

At Monks, for instance, inference costs have moved from scattered pilots into formal planning. The company updates its annual budgets with “very clear expectations for our token budgets,” said co-founder and chief AI officer Wesley ter Haar. What had been background experimentation is now a “line item,” he added. Those costs are shaping vendor negotiation and even internal infrastructure strategy including discussions about whether to invest in their own “AI factory” or rely on external cloud models. 

Could tokens influence how agencies negotiate deals with platforms?

Yes.

Once inference becomes a real cost center, it stops being a back-end technical issue and starts entering commercial conversations. Agencies are already updating budgets and vendor relationships as AI usage moves from pilots to ongoing workflows. That shift puts volume on the table. Agencies controlling large amounts of AI workload have something tangible to negotiate with: how much inference they buy, where it runs and under what terms. Over time, compute could play a similar role to media volume in broader negotiations — folded into service agreements, preferred pricing discussions or bundled partnerships with platforms and AI providers. Access to infrastructure, pricing tiers or model capabilities becomes part of the package, not just the backdrop.

Will advertisers ever pay for tokens? 

Not likely because they’re not a value metric. They’re a production input. 

“Marketing organizations would rather pay for outcomes than something like tokens, which they really don’t understand the value of and have really difficult time forecasting,” said Gartner’s Frank. “Agencies ought to be better at forecasting, because they have a lot more experience, and they can aggregate and consolidate a lot of that cost.”

That pushes the industry toward a model where tokens are abstracted away, much like server cycles in cloud computing. The cost still exists but it sits inside the service rather than being itemized.

What’s the arbitrage opportunity?

Anytime a new metered input enters advertising — one that CMOs don’t fully understand — commercial gray zones follow. Especially, at a time when the costs of tokens are so cheap while demand is on the rise. Agencies could: secure volume discounts, shift workloads between premium and cheaper models, run some inference on owned infrastructure — all while presenting clients with a bundled AI service fee. 

The tension is familiar. Advertising has seen similar dynamics with programmatic supply paths, agency trading desks and principal media. Tokens risk becoming the AI era version of an opaque technical input. Or as Robert Webster, founder of AI marketing consultancy TAU Marketing Solutions, explained: “As soon as agencies start arbitraging tokens, CMOs will lose transparency and we’re back to the same problems we currently have.”