Best LLM for campaign performance analysis

What the model is actually being asked to do

The current test pack contains seven synthetic datasets, each with a known performance finding. One has a standout ad whose results are driven by a two-day spike in Reels. Another has an unusual single-day spend surge. A third shifts spend towards a campaign attracting more post-view conversions, while its post-click performance does not justify the increase.

The model receives the spreadsheet data and an analytical brief: use CPA and ROAS, identify the main finding, explain it and recommend what to change. Each dataset is also tested separately with a scaffold, a general checklist covering aggregation, breakdowns, unreliable data and one-off effects. Neither setup tells it which finding has been planted.

Scores measure recovery of the expected analysis, rather than writing quality or a human skill level. How the test is scored →

The breakdowns are part of the analysis

The encouraging result is that the leading models can recover much of the intended story from these datasets. However, the test supplies the information needed to find it. It does not test whether the model can obtain the right reports from a live account.

That distinction matters. A monthly ad total does not tell you whether its advantage lasted all month or came from two exceptional days on one placement. Combining post-click and post-view conversions into a single number removes the distinction needed to investigate an attribution shift.

These are decisions you make when preparing the export. A better model cannot reconstruct a breakdown that is absent from the information you supply. It can ask for it, which is itself useful, but the workflow then needs a way to provide it.

Build the marketer’s process around the model

My view is that a skilled performance marketer should be able to get considerably more from this by designing the inputs and the investigation. The existing checklist is a start; there is room to build a workflow that retrieves the right breakdowns, checks the calculations and follows up on what it finds.

For example, a proposed workflow could examine a winning ad by day and placement, then recalculate its performance without an exceptional period before recommending more budget. A shift towards post-view conversions could trigger a comparison of post-click trends and a request for evidence that the additional spend is producing additional sales. The instructions would define those checks and the evidence needed to act.

That is the improvement I would test next. A high success rate is a reasonable ambition, however this benchmark has not established one for that fuller workflow. It needs testing on fresh cases, including ordinary periods where there is no anomaly to find. Otherwise, we risk building a process that is very good at finding the answers we have already told it to expect.

Use the benchmark to choose a model worth trying. Then apply your marketing expertise to what it can see, where it should investigate and what would justify its recommendation. Those decisions belong in the workflow you build once and improve as you use it.