Matalan’s AI search ads are performing better than ones written by humans

The tactic was considered so promising that Matalan has begun piloting it on other product categories, including children’s shoes, homewares and womenswear. ”We find Google Shopping to be one of the best ways to drive our online performance and ensure that our products are reaching customers,” said Ali Jones, Matalan’s chief customer and omnichannel officer, in an email.

The product descriptions were used in Google Shopping ads — promoted search results that show up in the Google Shopping tab. Lucy Barnes, general manager at Havas Media Manchester, told Digiday that channel was currently the largest source of search traffic to Matalan’s website, though she didn’t say what proportion of its overall traffic Google Shopping accounted for.

“Google Shopping is huge for us and is our biggest driver of performance, so any kind of effectiveness, any kind of efficiencies or increased results that we can get within the search engine results page is brilliant,“ she said.

Matalan has experimented with AI copy previously, using the Gemini and PaLM 2 models to write product descriptions on its own site. For Matalan’s Google Shopping ads, Havas Media deployed a solution that relies upon ChatGPT-4.

The media agency compared impressions driven by AI-augmented product ads for 1,100 items in Matalan’s men’s suits range, with 3,000 other menswear items with product ads written by people. The test ran for 31 days between March and April, a time when the brand didn’t have other promotional spend already planned. Although Matalan releases new collections each year, demand for suits remains generally consistent (as Jones describes it as a “strong volume” vertical with “less volatility” than other product categories), making it a suitable category to use as a sample.

Based on Havas and Matalan’s four-week test, Barnes said the agency’s data science team calculated that the AI-written product ads drove over 11,900 more impressions per week than ads with product copy written without AI — equivalent to a 16% increase per £1 ($1.28) spent.

According to Barnes, the AI copy was favored over human-written descriptions by Google’s system, making Matalan’s products more visible than competitors. “The AI-driven feed descriptions pick up on more relevant product attributes than a human-written product description, which improves the products visibility in the search engine and ultimately drives more impressions per product,” she said.

Comparing Matalan’s human-written ads with ones written by AI makes the difference clear. The former run to about 100 words and take plenty of space and adjectives to describe a product, while the latter feature short sentences studded with search keywords.

“The category is competitive. To be able to get some enhancement, whether it’s impressions or visibility on that is, is superb,” Barnes added.

That AI-written copy is favored by ranking algorithms isn’t necessarily a surprise. Craig Elimeliah, chief creative officer of digital agency Code & Theory, told Digiday that “computers talk to computers much better than humans can talk to computers.”

“Anyone writing SEO copy at this point is antiquated, to be honest. I mean, that is the perfect job for AI,” he added.

Understanding whether the AI-written ads drove more traffic required comparing them to a control group. Rather than run some suiting ads without AI intervention, Barnes said the team used the rest of Matalan’s menswear category to provide a benchmark for search impressions.

Though that method isn’t as comprehensive as a traditional A/B test, it’s a quicker approach that allows for adjustments on the fly. Will Hanschell, chief executive officer of AI software-as-a-service company Pencil, told Digiday that although A/B tests can provide a higher standard of evidence, approaches like the one taken by Matalan and Havas can still yield legitimate results.

“It’s expedient, it’s a convenient way of getting a test actually done in the real world,” he said. Pencil has run similarly structured pilot tests on ads containing AI-generated copy for Amplifon, a hearing aid supplier; the agency found AI ads reduced the brand’s cost-per-lead by 20% over a three-month test period. Hanschell did not provide exact figures.

Hanschell said that forgoing A/B tests can also allow marketers to act on positive results faster. “It can be a better and preferred approach in some cases, like if you’re a B2C, because you can act on the test immediately. It’s working, spend more. You don’t have to wait for the A/B test [to finish] and then create new assets that take the learnings from that test.”

Matalan is the only Havas client to have used the AI solution, which it’s dubbed ”Performance Feed Optimizer.”

Barnes said that though conversion rates held steady, higher traffic driven by increased search impressions meant the volume of conversions on the Matalan site rose, though she didn’t share the size of that increase. Matalan’s marketing team plans on rolling out AI-written product ads in the future. “It’s a proof of concept that works,” said Barnes.

”It is certainly an area we’ll be looking at closely as we optimize our shopping strategy. We intend to take full advantage of our innovative approach to AI-powered feed management and expect to lead the market in this area,” added Jones. ”Our ambition is to see similar results as we scale Havas’ Media AI feed optimization tool across other product categories, which will in turn drive significant efficiencies for our business.”

Editor’s Note: This article originally stated Havas and Matalan had not analyzed conversion rates. Havas later clarified that analysis had taken place.