Campaign insight memory: benchmark methods and results
Models extract reusable learning from synthetic campaign results and update an existing insight library when new evidence arrives. The scorer checks genuine signals, misleading patterns, confidence and the correct memory-update decisions.
Reviewed 16 October 2026 · Tests and editorial direction: Ben Nancarrow · AI-assisted writing and checks.
The signal-extraction pack contains 22 ads with known signals, decoys, low-evidence traps and a data-quality issue. The memory-update pack contains six existing insights and a ten-ad follow-up digest with eight decision points. These are two controlled cases; the model rows have different coverage and repeat counts.
What changes between prompts
The baseline states the task and output shape. The scaffold adds evidence floors, a confound checklist, calibration rules and an update vocabulary. Both use the same campaign evidence; the evaluator-only answers are withheld.
How the result is scored
The deterministic composite checks signal recovery, false insights, confidence, traps and memory updates against the case key. It rewards retaining, narrowing or replacing a learning when the evidence warrants it. It does not score fluent summary writing or measure a downstream campaign lift.
All 17 reviewed model results
Deterministic insight fidelity / 10
Model
Result / setup
Scored outputs
Configuration / recorded test date
Cost / scored output
Gemini 3.5 Flashprovisional
8.25/10Scaffolded prompt
6
reasoning_high4 July 2026
$0.011Recorded spend
Meta Muse Wonderprovisional
7.71/10Scaffolded prompt
4
max12 October 2026
$0.386Recorded spend
Gemini 3.7 Flashprovisional
7.57/10Scaffolded prompt
12
reasoning_high20 August 2026
$0.009Recorded spend
Claude Opus 5.5provisional
7.56/10Scaffolded prompt
4
thinking_max23 September 2026
$2.022Mixed cost basis
Claude Fable 5provisional
7.44/10Scaffolded prompt
12
thinking_max20 August 2026
$1.080API-equivalent estimate
Google Gemini 4 Argonprovisional
7.44/10Scaffolded prompt
4
high13 October 2026
$0.282Recorded spend
Meta Muse Spark 1.3provisional
7.44/10Scaffolded prompt
4
max8 September 2026
$0.093Recorded spend
Gemini 3.6 Flashprovisional
6.99/10Combined setups
12
reasoning_high15 August 2026
$0.021Recorded spend
Gemini 3.1 Pro Previewprovisional
6.76/10Combined setups
12
Not recorded20 August 2026
Not recorded
GPT-6 Solprovisional
6.65/10Simple prompt
4
xhigh23 September 2026
$0.098Recorded spend
Claude Opus 5provisional
6.51/10Combined setups
12
thinking_max15 August 2026
$0.498Recorded spend
GPT-6 Astraprovisional
6.50/10Scaffolded prompt
2
xhigh5 September 2026
$0.595Recorded spend
GPT-5.6 Solprovisional
5.77/10Combined setups
12
xhigh15 August 2026
$0.219Recorded spend
GPT-5.5provisional
5.73/10Scaffolded prompt
8
reasoning_high, reasoning_medium3 July 2026
$0.037Recorded spend
Claude Opus 4.8provisional
5.64/10Combined setups
6
thinking_high3 July 2026
$0.243Recorded spend
Claude Fable 5.1provisional
5.62/10Scaffolded prompt
2
thinking_max1 September 2026
Not recorded
Gemini 3.8 Flashprovisional
5.25/10Scaffolded prompt
2
reasoning_high4 September 2026
$0.061Recorded spend
Recorded output counts can include repeats and both prompt conditions. Configurations and cost bases are retained; joint scores remain joint.
This record preserves the evidence and interpretation reviewed together. The live explorer can advance independently. Public results include provisional evidence; the methods above define what the scores measure.