Marketers have a new audience to worry about — large language models

With AI platforms like ChatGPT and Perplexity gaining traction, tech firms are creating new ways to understand how large language models perceive their brands — and what AI answers say about them. One of the newest developments comes from Profound, an SEO startup focused on AI search platforms, which today is debuting a way to estimate conversation volume rather than just analyzing AI outputs.

Just like Google provides data about traditional search volume and trends, Profound hopes its updates will identify user intent when talking with chatbots — something that’s still not possible with the current black-box nature of generative search. One way brands might use Profound’s dashboard: To inform how they create, test and optimize content based on what people are chatting with bots about on various platforms. It also could help them better track the competitive landscape based on the companies people mention when using generative search.

“Over a 30-day period, if you ask 21,000 questions to the major AI answer engines about fast food and McDonald’s shows up in 50% for them, that’s useful,” Profound CEO and co-founder James Cadwallader told Digiday. “Brands are very interested in that. They’re paying for that. McDonald’s is a very important piece of data for them. But what that does not answer is how many people on the internet are searching for fast food in these AI answer engines.”

If there’s a dataset about queries about for example, “Toshiba microwaves,” Cadwallader said Profound’s predictive model can estimate how many searches mentioned other microwave brands. He wouldn’t disclose Profound’s sources for external data, but said they’re opt-in, privacy compliant and that it’s still in beta.

To analyze meaning and context of chats, Profound uses billions of semantic embeddings in real time and then uses a custom sampling algorithm to identify conversation patterns with statistical significance. It also developed new approaches to temporal query optimization since traditional methods break down when dealing with the asynchronous nature of AI conversation flows.

Some SEO experts like Kevin Indig already see the benefits of using Profound’s volume of data, API access and citations within Profound are all appealing. He said brands could use it to create or optimize content based on what people are chatting with bots about on various platforms. One example Indig gave was his client Hims, which is in the process of using Profound’s platform to fill content gaps or improve existing materials.

“We’re noticing there’s a lot of demand around topics related to diet programs,” said Indig, an advisor to brands’ growth teams. People are searching for the best diet or what’s the easiest diet, but in more elaborate ways than on Google. We realized that even though we have weight loss content, which is a massive topic for Hims, we don’t specifically compare diet programs.”

Though the solutions differ in the details, each aims to help marketers understand better how LLMs such as Gemini, ChatGPT and Meta’s Llama are portraying their brands. Profound isn’t alone in its roll out. Brandtech-owned marketing firm Jellyfish debuted its own solution, “Share of Model,” earlier this month. Jellyfish’s initial clients using the new tool include Danone and Pernod Ricard whisky brand Chivas Brothers.

Understanding search behavior

Changes in search user behavior required “a totally different way to think about optimization,” said Jack Smyth, chief solutions officer for AI, planning and insights at Jellyfish. His company’s tool uses APIs from various platforms for a “birds-eye view” of LLM brand portrayals across video, image and text assets used to train AI models.

Users can also upload creative assets — catalogs, websites, or social media campaigns — to evaluate how well they align with how each AI model thinks about various categories and concepts. That could help drive product recommendations as AI search platforms build new features for e-commerce. “Share of Model” also integrates with other Brandtech tools for AI content creation like Pencil Pro.

“Our point of view is that everything that you post across any platform is now part of someone else’s training set,” Smyth said. “Every asset that an advertiser creates is now a brief to someone else’s model.”

Web users are increasingly turning away from traditional search or browsing tools in favor of AI “agents,” such as chatbots. 66% of 18-24 year olds and 51% of 25-34 year olds regularly refer to AI tools for product recommendations, according to YouGov. 

As usage increases, so do the stakes. Platforms like ChatGPT and Google’s AI Overviews base answer queries by searching vast amounts of information pulled from the broader internet. However, aggregated responses might not line up with a brand’s carefully constructed market positioning — and might be outright wrong.

“Consumers are seeking clarity in a sea of options. As brands, we must ensure our products are represented in those critical AI-driven responses,” said Gokcen Karaca, head of digital and design at Chivas Brothers, a whisky brand owned by Pernod Ricard.

AI applications are high on marketers’ priorities going into the new year. A recent survey of over 700 marketers by Serviceplan Group, the German agency network, found 81% said AI was their top priority — above brand-building, measuring marketing ROI or CRM investment.

Why LLMs’ influence matters for marketers

“LLMs will increasingly influence customer behavior, and the Share of Model platform enables us to track and compare each LLM’s perception,” Catherine Lautier, vp and global head of media and integrated brand communication at Danone said in a statement.

Other firms that have developed similar tools include Hubspot, which launched free “AI Search Grader” tool in August and recently added Perplexity to its list of AI platforms that can be analyzed. Another is BrightEdge, an SEO marketing tech company that recently rolled out a way for brands to measure how they show up in Google’s AI Overviews.

At $72,000 a year — the floor price for an annual subscription of Jellyfish’s Share of Model Platform — these solutions are just a fraction of what some companies are spending on overall AI investments. Coca-Cola, for example, committed to investing over $1 billion in Microsoft’s AI solutions in April.

There’s an opportunity cost to consider, Smyth said. Ignoring how LLMs were interacting with a brand’s digital output, and the way they might be influencing potential customers, could cause problems as generative AI tool usage becomes more widespread. 

“People might be less likely to interact in a chat interface, but they might be using an agent to do their shopping,” Smyth said. “Or you might have to think about building a website that isn’t primarily for humans; it’s actually designed as a repository of information for models to go and make proactive recommendations on different platforms.”