The full benmark
See what AI can do to human level or better.
Use it to decide which tasks to automate, how much time you should save, which model to use and where you get the best results from different prompts/scaffolds.
What these totals count
Scored run records on the homepage count distinct scored records in Benmark's scoring history. A record can represent a run or a batch of candidate outputs.
Test runs tracked in the explorer count testing activity grouped by run, including scored work, work awaiting scoring, and blocked or unfinished runs. A run used for several tasks is counted once in this total.
The two views select and group records differently, so subtracting their totals does not give the number of unscored runs. Neither total counts independent datasets or individual model answers. Use each task's sample counts to judge the evidence behind a result.
Three practical decisions
What to automate. Which model. Whether a scaffold helps.
Start with the overview. Open a task to see what was tested, my take and the scores.
Current published task results. Compare models within a task: human assessments and automated answer-key scores measure different things.
Reference ad to new concept
Take a reference ad and turn what makes it interesting into a new concept for a different brand.
Highest published task score: 8.33/10 — Google Gemini 4 Argon. Provisional evidence.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Google Gemini 4 ArgonProvisional 8.33Simple prompt $0.031recorded spend Gemini 3.8 FlashProvisional 7.89Simple prompt $0.009recorded spend GPT-6 SolProvisional 7.89Simple prompt $0.024recorded spend Gemini 3.1 Pro PreviewProvisional 7.89Simple prompt $0.026recorded spend Claude Opus 5.5Provisional 7.72Simple prompt Not recorded Gemini 3.5 FlashProvisional 7.56Simple prompt $0.004recorded spend Claude Fable 5.1Provisional 7.50Scaffolded prompt $0.448API-equivalent estimate Claude Fable 5Provisional 7.43Simple prompt $0.172API-equivalent estimate GPT-5.5Provisional 7.41Simple prompt $0.022recorded spend Meta Muse WonderProvisional 7.39Simple prompt $0.097recorded spend Meta Muse Spark 1.3Provisional 7.33Simple prompt $0.019recorded spend Gemini 3.7 FlashProvisional 7.30Simple prompt $0.003recorded spend GPT-6 AstraProvisional 7.28Simple prompt $0.199recorded spend Claude Opus 4.8Provisional 7.26Simple prompt $0.034recorded spend GPT-5.6 SolProvisional 7.11Simple prompt $0.048recorded spend Claude Opus 5Provisional 6.92Simple prompt Not recorded Inspect this task · Read the findings →
How we test this
Models turn reference ads into new concepts for the same synthetic brand. The fixed source set contains three images and three videos; the image-only mode uses the three images. The concepts are blind-scored for commercial potential, originality and suitability.
What the model receives
Each call supplies one source ad and brand context for an invented brand, then asks for one new concept. The aim is to adapt the useful creative mechanism to the new product. The full media mode covers six source ads; output counts can also include repeats and both prompt conditions.
What changes between prompts
Both conditions receive the same brand context and line-specific inputs, with a 150-word concept limit. The simple prompt states the task. The scaffold adds judging criteria, an ad-quality framework and a hook/narrative/visual/conversion structure. It therefore tests that whole package of guidance, not a single wording change.
How the result is scored
Ben blind-scores commercial potential, creative innovation, and coherence/suitability on a 0–10 scale. A concept clears the test-worthy threshold only when every dimension reaches 8. The displayed mean summarises quality; it does not measure the ad's eventual conversion rate. Scoring hides model labels, although the scaffold's formatting can reveal the prompt condition.
Brand context to concepts
Come up with ad concepts using just the brand, product and audience brief.
Highest published task score: 8.11/10 — Claude Opus 5.5. Provisional evidence.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Claude Opus 5.5Provisional 8.11Simple prompt Not recorded Gemini 3.1 Pro PreviewProvisional 7.33Scaffolded prompt $0.010recorded spend Google Gemini 4 ArgonProvisional 7.22Simple prompt $0.012recorded spend Meta Muse Spark 1.3Provisional 6.84Scaffolded prompt $0.007recorded spend GPT-6 SolProvisional 6.78Simple prompt $0.010recorded spend Claude Fable 5.1Provisional 6.67Scaffolded prompt $0.250API-equivalent estimate Gemini 3.7 FlashProvisional 6.63Scaffolded prompt $0.001recorded spend Claude Opus 4.8Provisional 6.59Simple prompt $0.011recorded spend Gemini 3.5 FlashProvisional 6.56Scaffolded prompt $0.001recorded spend Meta Muse WonderProvisional 6.56Simple prompt $0.027recorded spend Gemini 3.8 FlashProvisional 6.33Simple prompt $0.004recorded spend GPT-5.6 SolProvisional 6.33Scaffolded prompt $0.015recorded spend GPT-5.5Provisional 6.30Simple prompt $0.005recorded spend GPT-6 AstraProvisional 6.11Simple prompt $0.089recorded spend Claude Fable 5Provisional 6.00Scaffolded prompt $0.094API-equivalent estimate Claude Opus 5Provisional 5.78Simple prompt Not recorded Inspect this task · Read the findings →
How we test this
Models receive the synthetic brand's product and audience context and generate three concepts per prompt condition. Ben scores whether those concepts are commercially promising, distinctive and coherent enough to test.
What the model receives
The model receives brand guidelines for an invented brand without a reference ad. It generates three numbered concepts from that context. The six outputs in a complete baseline/scaffolded run are three concepts per condition, not six brands or campaigns.
What changes between prompts
Both conditions receive the same brand context and line-specific inputs, with a 150-word concept limit. The simple prompt states the task. The scaffold adds judging criteria, an ad-quality framework and a hook/narrative/visual/conversion structure. It therefore tests that whole package of guidance, not a single wording change.
How the result is scored
Ben blind-scores commercial potential, creative innovation, and coherence/suitability on a 0–10 scale. A concept clears the test-worthy threshold only when every dimension reaches 8. The displayed mean summarises quality; it does not measure the ad's eventual conversion rate. Scoring hides model labels, although the scaffold's formatting can reveal the prompt condition.
Insights and ads to a new proposal
Use what we've learnt from a campaign, alongside its existing ads, to suggest what to test next.
Highest published task score: 7.67/10 — Claude Opus 5.5. Provisional evidence.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Claude Opus 5.5Provisional 7.67Simple prompt Not recorded Google Gemini 4 ArgonProvisional 7.33Simple prompt $0.018recorded spend Gemini 3.1 Pro PreviewProvisional 7.22Scaffolded prompt $0.011recorded spend Meta Muse Spark 1.3Provisional 7.00Scaffolded prompt $0.007recorded spend Claude Fable 5.1Provisional 7.00Simple prompt $0.415API-equivalent estimate Gemini 3.8 FlashProvisional 6.89Simple prompt $0.004recorded spend Gemini 3.5 FlashProvisional 6.85Simple prompt $0.001recorded spend Claude Opus 5Provisional 6.78Simple prompt Not recorded Gemini 3.7 FlashProvisional 6.74Simple prompt $0.001recorded spend Claude Opus 4.8Provisional 6.74Simple prompt $0.013recorded spend GPT-5.5Provisional 6.67Simple prompt $0.008recorded spend GPT-6 SolProvisional 6.56Simple prompt $0.015recorded spend Meta Muse WonderProvisional 6.56Simple prompt $0.026recorded spend GPT-5.6 SolProvisional 6.41Simple prompt $0.037recorded spend GPT-6 AstraProvisional 6.22Scaffolded prompt $0.122recorded spend Claude Fable 5Provisional 6.11Simple prompt $0.135API-equivalent estimate Inspect this task · Read the findings →
How we test this
Models receive brand context, codified campaign learning and a digest of the existing creative portfolio, then propose three next tests per prompt condition. The proposals are blind-scored for their commercial and creative quality.
What the model receives
The inputs combine brand guidelines for an invented brand, existing insight memory and a live-ad gap digest. Each condition asks for three concepts. This is a test of translating supplied learning into proposals; the model is not discovering that learning from a live account.
What changes between prompts
Both conditions receive the same brand context and line-specific inputs, with a 150-word concept limit. The simple prompt states the task. The scaffold adds judging criteria, an ad-quality framework and a hook/narrative/visual/conversion structure. It therefore tests that whole package of guidance, not a single wording change.
How the result is scored
Ben blind-scores commercial potential, creative innovation, and coherence/suitability on a 0–10 scale. A concept clears the test-worthy threshold only when every dimension reaches 8. The displayed mean summarises quality; it does not measure the ad's eventual conversion rate. Scoring hides model labels, although the scaffold's formatting can reveal the prompt condition.
Iterate on a top performer
Suggest a new version of a successful ad. We score the recommendation, not a finished ad or its future performance.
Highest published task score: 9.47/10 — Claude Opus 5. Provisional evidence.
Compare 13 models
Published model scores Model Score / setup Cost per scored output Claude Opus 5Provisional 9.47Scaffolded prompt $0.118recorded spend Claude Opus 5.5Provisional 9.43Scaffolded prompt $0.573API-equivalent estimate Google Gemini 4 ArgonProvisional 9.33Simple prompt $0.034recorded spend Gemini 3.7 FlashProvisional 9.22Scaffolded prompt $0.005recorded spend Meta Muse WonderProvisional 9.03Simple prompt $0.082recorded spend Gemini 3.5 FlashProvisional 9.01Scaffolded prompt $0.005recorded spend Gemini 3.8 FlashProvisional 8.87Simple prompt $0.009recorded spend Claude Fable 5.1Provisional 8.83Scaffolded prompt $0.441API-equivalent estimate Gemini 3.1 Pro PreviewProvisional 8.78Scaffolded prompt $0.007recorded spend Meta Muse Spark 1.3Provisional 8.70Scaffolded prompt $0.016recorded spend Gemini 3.1 Flash-LiteProvisional 8.23Scaffolded prompt $0.001recorded spend GPT-6 SolProvisional 8.23Scaffolded prompt $0.022recorded spend GPT-6 AstraProvisional 8.03Scaffolded prompt $0.147recorded spend Inspect this task · Read the findings →
How we test this
Using five image ads and five video ads, models recommend how to develop a stronger next version. Ben scores the recommendation separately from the explanation of the original ad.
What the model receives
The fixed pack contains five static images and five videos. Models see the media, with original filenames and account performance withheld. They receive no spend, conversion, reach or brand-context evidence. Analysis and creative recommendations are generated as separate outputs.
What changes between prompts
Each asset is tested with a simple and a scaffolded prompt. The analysis scaffold covers attention, product relevance, proof, platform fit and other creative mechanisms. The action scaffold asks what to preserve, what to change and why. The media remains identical between conditions.
How the result is scored
Action outputs are judged on the recommendation's creative and commercial quality, including coherence/commercial judgment as a critical dimension. These are ratings of the proposed next version. Neither the generated ad nor its subsequent campaign performance is tested in this task.
Idea scoring and priority plan
Turn campaign results and creative ideas into a production plan: what to make next, and what to leave out.
Highest published task score: 10.00/10 — Gemini 3.7 Flash, Gemini 3.8 Flash and 8 other models. Provisional evidence.
Equal top scores do not establish equivalent performance. Cost figures may use different bases or pool prompt setups.
Compare 17 models
Published model scores Model Score / setup Cost per scored output Gemini 3.7 FlashProvisional 10.00Scaffolded prompt $0.014recorded spend Gemini 3.8 FlashProvisional 10.00Scaffolded prompt $0.049recorded spend Meta Muse Spark 1.3Provisional 10.00Scaffolded prompt $0.085recorded spend GPT-6 SolProvisional 10.00Simple prompt $0.110recorded spend Google Gemini 4 ArgonProvisional 10.00Simple prompt $0.122recorded spend Meta Muse WonderProvisional 10.00Simple prompt $0.238recorded spend GPT-6 AstraProvisional 10.00Simple prompt $0.577recorded spend Claude Opus 4.8Provisional 10.00Simple prompt $1.170recorded spend Claude Opus 5.5Provisional 10.00Scaffolded prompt $1.626API-equivalent estimate Claude Fable 5.1Provisional 10.00Simple prompt $2.005API-equivalent estimate Gemini 3.1 Pro PreviewProvisional 9.92Scaffolded prompt $0.093recorded spend Claude Opus 5Provisional 9.92Combined setups $0.393recorded spend GPT-5.5Provisional 9.83Combined setups $0.154recorded spend Claude Fable 5Provisional 9.83Scaffolded prompt $1.080API-equivalent estimate Gemini 3.6 FlashProvisional 9.75Combined setups $0.027recorded spend GPT-5.6 SolProvisional 9.75Combined setups $0.184recorded spend Gemini 3.5 FlashProvisional 9.36Simple prompt $0.015recorded spend Inspect this task · Read the findings →
How we test this
Models build a six-item production queue from one synthetic brand-cycle pack: four video slots and two static slots. The answer key checks whether the plan follows the performance evidence, rejects misleading gaps and respects production constraints.
What the model receives
The pack combines brand context, production constraints, the current creative portfolio, a performance table, a performance-blind diversity report and a codified insight library. It deliberately includes useful targets, contradicted gaps, weak-evidence traps and a data-quality exclusion.
What changes between prompts
Both prompts include the complete pack and JSON output contract. The scaffold adds a method for reconciling apparent gaps with performance and existing learning. It does not add the evaluator's answers.
How the result is scored
A deterministic score combines target recall, decoy rejection, trap handling, insight linkage and ordering. Repeated runs reuse this one brand-cycle case. A 10/10 means the plan satisfied this test's key; the score is not a rating of a finished ad's commercial impact.
Hook development
Turn an ad concept into opening hooks that catch attention and give someone a reason to keep watching.
Highest published task score: 8.75/10 — Meta Muse Spark 1.3. Published evidence.
Compare 19 models
Published model scores Model Score / setup Cost per scored output Meta Muse Spark 1.3Published 8.75Scaffolded prompt Not recorded Claude Opus 5Published 8.50Scaffolded prompt Not recorded Gemini 3.7 FlashPublished 8.50Scaffolded prompt Not recorded Google Gemini 4 ArgonPublished 8.25Scaffolded prompt Not recorded Claude Fable 5Published 8.00Scaffolded prompt Not recorded Claude Opus 4.7Provisional 8.00Scaffolded prompt Not recorded Gemini 3.1 Pro PreviewProvisional 8.00Scaffolded prompt Not recorded GPT-6 AstraPublished 8.00Scaffolded prompt Not recorded Claude Opus 4.8Provisional 7.83Scaffolded prompt Not recorded Gemini 3.1 Flash-LiteProvisional 7.83Simple prompt Not recorded GPT-5.6 SolPublished 7.75Simple prompt Not recorded GPT-5.4 MiniProvisional 7.58Scaffolded prompt Not recorded GPT-5.5Provisional 7.58Simple prompt Not recorded Meta Muse WonderPublished 7.50Simple prompt Not recorded Gemini 3.8 FlashPublished 7.00Simple prompt Not recorded Claude Fable 5.1Published 6.75Scaffolded prompt Not recorded Claude Opus 5.5Published 6.75Simple prompt Not recorded GPT-6 SolPublished 6.75Simple prompt Not recorded Gemini 3.5 FlashPublished 6.29Scaffolded prompt Not recorded Inspect this task · Read the findings →
How we test this
Models develop opening hooks from a fixed concept and brand context. Ben blind-scores attention, commercial sharpness, relevance and product specificity, allowing a catchy line to be distinguished from a useful selling idea.
What the model receives
A sparse concept and its commercial context are supplied from the active concept-to-brief pack. Current rows have different repeat coverage; the results table records output counts. A pair of outputs can represent the same concept in two prompt conditions.
What changes between prompts
The same fixed creative input and brand context are used with plain and scaffolded-context prompts. The structured condition adds guidance for developing the output. The publisher keeps hooks, narratives and compliant briefs as separate tasks, with their own scores.
How the result is scored
Each hook receives four human ratings: attention grab, commercial sharpness, relevance and product specificity. The mean forms the task score. The scale is anchored to useful paid-social work, with 8 representing the skilled-human usability threshold. It does not measure live hook rate or conversion rate.
Narrative development
Build the story of an ad from a concept and hook, keeping the product and audience in mind.
Highest published task score: 8.58/10 — Claude Opus 4.7. Provisional evidence.
Compare 19 models
Published model scores Model Score / setup Cost per scored output Claude Opus 4.7Provisional 8.58Simple prompt Not recorded Claude Opus 5Published 8.50Scaffolded prompt Not recorded Claude Opus 4.8Provisional 8.17Simple prompt Not recorded Claude Fable 5Published 8.00Simple prompt Not recorded Claude Fable 5.1Published 8.00Scaffolded prompt Not recorded GPT-5.5Provisional 8.00Simple prompt Not recorded GPT-5.6 SolPublished 8.00Scaffolded prompt Not recorded Gemini 3.1 Pro PreviewProvisional 7.83Simple prompt Not recorded Claude Opus 5.5Published 7.75Simple prompt Not recorded Gemini 3.1 Flash-LiteProvisional 7.75Scaffolded prompt Not recorded Gemini 3.7 FlashPublished 7.75Simple prompt Not recorded Gemini 3.8 FlashPublished 7.75Simple prompt Not recorded Google Gemini 4 ArgonPublished 7.75Simple prompt Not recorded GPT-5.4 MiniProvisional 7.75Simple prompt Not recorded GPT-6 AstraPublished 7.50Scaffolded prompt Not recorded GPT-6 SolPublished 7.50Simple prompt Not recorded Meta Muse Spark 1.3Published 7.50Scaffolded prompt Not recorded Gemini 3.5 FlashPublished 6.86Scaffolded prompt Not recorded Meta Muse WonderPublished 6.50Scaffolded prompt Not recorded Inspect this task · Read the findings →
How we test this
Models develop an ad's story from the fixed creative input and brand context. Ben blind-scores coherence, commercial persuasion, emotional levers and product relevance.
What the model receives
The concept-to-brief pack fixes the creative starting point so models can be compared on developing the narrative. Coverage differs by model and repeat count; repeated versions of this pack do not represent independent audiences or brands.
What changes between prompts
The same fixed creative input and brand context are used with plain and scaffolded-context prompts. The structured condition adds guidance for developing the output. The publisher keeps hooks, narratives and compliant briefs as separate tasks, with their own scores.
How the result is scored
The narrative score averages coherence, commercial persuasion, use of emotional levers and product relevance. This measures the written narrative. The hook and the production handover have separate task scores, so a high narrative result is not inherited by the whole workflow.
Compliant brief generation
Turn an agreed concept into a production brief that follows the brand rules and tells the team what to make.
Highest published task score: 9.50/10 — Claude Fable 5, Claude Fable 5.1 and 8 other models. Published evidence.
Equal top scores do not establish equivalent performance. Cost figures may use different bases or pool prompt setups.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Claude Fable 5Published 9.50Simple prompt Not recorded Claude Fable 5.1Published 9.50Simple prompt Not recorded Claude Opus 5Published 9.50Scaffolded prompt Not recorded Claude Opus 5.5Published 9.50Simple prompt Not recorded Gemini 3.8 FlashPublished 9.50Scaffolded prompt Not recorded GPT-5.6 SolPublished 9.50Scaffolded prompt Not recorded GPT-6 AstraPublished 9.50Scaffolded prompt Not recorded GPT-6 SolPublished 9.50Simple prompt Not recorded Meta Muse Spark 1.3Published 9.50Scaffolded prompt Not recorded Meta Muse WonderPublished 9.50Simple prompt Not recorded Claude Opus 4.8Provisional 9.39Scaffolded prompt Not recorded GPT-5.5Published 9.38Scaffolded prompt Not recorded Gemini 3.1 Pro PreviewPublished 9.37Scaffolded prompt Not recorded Gemini 3.7 FlashPublished 9.17Scaffolded prompt Not recorded Google Gemini 4 ArgonPublished 9.17Simple prompt Not recorded Gemini 3.5 FlashPublished 8.81Simple prompt Not recorded Inspect this task · Read the findings →
How we test this
Models turn a fixed concept and brand/production constraints into a structured brief. The public task score measures brief quality with a compliance gate; hook and narrative quality are assessed separately.
What the model receives
The input supplies the creative starting point, brand context and production constraints. The output should give production a grounded, actionable handover. Recorded model rows can combine more than one completed cycle; the results table preserves their output counts.
What changes between prompts
The same fixed creative input and brand context are used with plain and scaffolded-context prompts. The structured condition adds guidance for developing the output. The publisher keeps hooks, narratives and compliant briefs as separate tasks, with their own scores.
How the result is scored
The published compliant-brief task uses the brief-quality component and applies compliance as a gate. Its scorer is AI-assisted within the reviewed scoring process. Human hook and narrative scores exist in the wider brief evaluation but are deliberately excluded from this task's headline. A 9.5 here therefore describes the handover, not a 9.5 advertising idea.
Static image generation
Make a basic static ad from a brief and product reference. We score the image as generated, before any editing.
Highest published task score: 8.84/10 — GPT Image 2.5 Sunburst. Provisional evidence.
Compare 4 models
Published model scores Model Score / setup Cost per scored output GPT Image 2.5 SunburstProvisional 8.84Scaffolded prompt Not recorded Seedream 5 ProProvisional 7.64Simple prompt Not recorded GPT Image 2Provisional 7.44Scaffolded prompt Not recorded Gemini 3 Pro ImageProvisional 7.12Simple prompt Not recorded Inspect this task · Read the findings →
How we test this
Image models create ads from five fixed briefs, using identical product-reference material in both prompt conditions. Ben scores the unedited images on fidelity, copy, claims, compliance and craft.
What the model receives
The image test uses five briefs and one canonical product-reference contact sheet. Each brief produces one image with the simple prompt and one with the scaffold, giving ten outputs per complete run. The reference bytes are shared across models and conditions.
What changes between prompts
Provider-neutral prompts keep the task and reference constant. The scaffold adds concise ad-anatomy guidance. Prompt length is bounded rather than silently truncated, so a model is assessed on the brief it actually received.
How the result is scored
Raw outputs are human-scored for product fidelity, copy accuracy, claims safety, brief compliance and craft quality. Each dimension must reach 8 for an output to clear the usability threshold. The displayed mean is a comparison measure; it can conceal a weak individual dimension. No editing pass is added before scoring.
Video generation
Make a basic eight-second vertical video ad from a brief. We score the full video as generated, before any editing.
Highest published task score: 6.68/10 — Gemini Omni Flash. Provisional evidence.
Compare 3 models
Published model scores Model Score / setup Cost per scored output Gemini Omni FlashProvisional 6.68Scaffolded prompt Not recorded Gen-4.5Provisional 5.24Scaffolded prompt Not recorded Seedance 2Provisional 4.56Scaffolded prompt Not recorded Inspect this task · Read the findings →
How we test this
Video models create complete eight-second vertical ads from five briefs. Both prompt conditions use text only, and Ben scores the unedited video; audio is excluded.
What the model receives
The briefs span UGC, founder, lifestyle, explainer and product-demonstration formats. The shared contract requests an eight-second 9:16 video at 720×1280. No reference images are supplied because providers interpret them differently. Five briefs in two conditions produce ten outputs per run.
What changes between prompts
The simple and scaffolded prompts share the same task. The scaffold adds concise ad-anatomy guidance within the common prompt limit. Video duration, geometry and decodability are checked before the result is assessed; scoring playback is muted.
How the result is scored
Raw outputs are human-scored for product fidelity, copy accuracy, claims safety, brief compliance and craft quality. Each dimension must reach 8 for an output to clear the usability threshold. The displayed mean is a comparison measure; it can conceal a weak individual dimension. No editing pass is added before scoring.
Content review
Review 10 images and 10 videos and spot the errors we've put in them, without being told what is wrong.
Highest published task score: 8.38/10 — Gemini 3.5 Flash. Provisional evidence.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Gemini 3.5 FlashProvisional 8.38Scaffolded prompt $0.001recorded spend Google Gemini 4 ArgonProvisional 8.00Simple prompt $0.012recorded spend Claude Opus 5.5Provisional 7.79Simple prompt $0.239API-equivalent estimate Meta Muse WonderProvisional 6.83Simple prompt $0.038recorded spend Gemini 3.1 Pro PreviewProvisional 6.79Scaffolded prompt $0.002recorded spend Gemini 3.7 FlashProvisional 6.68Scaffolded prompt $0.002recorded spend Claude Fable 5.1Provisional 6.38Simple prompt $0.312API-equivalent estimate Gemini 3.8 FlashProvisional 6.25Scaffolded prompt $0.011recorded spend Claude Opus 5Provisional 6.24Simple prompt $0.093recorded spend Claude Fable 5Provisional 5.50Scaffolded prompt $0.166API-equivalent estimate Meta Muse Spark 1.3Provisional 5.38Simple prompt $0.009recorded spend GPT-5.6 SolProvisional 5.22Scaffolded prompt $0.027recorded spend GPT-5.5Provisional 5.18Scaffolded prompt $0.039recorded spend Claude Opus 4.8Provisional 5.12Scaffolded prompt $0.028API-equivalent estimate GPT-6 AstraProvisional 5.12Scaffolded prompt $0.067recorded spend GPT-6 SolProvisional 4.96Scaffolded prompt $0.012recorded spend Inspect this task · Read the findings →
How we test this
Models review ten images and ten videos containing known defects. They are asked to check launch readiness without being told that a defect is guaranteed. Scoring measures how much of the answer-key defect set they find.
What the model receives
Both prompt conditions receive the same twenty media assets. The evaluator's key records the planted issue and observed secondary video defects. A complete run has forty reviews: twenty assets under each condition.
What changes between prompts
The simple prompt asks for launch QA; the scaffold adds a content-review checklist. The checklist directs the inspection without telling the model which defect is present in an asset.
How the result is scored
A single-defect asset scores 10 when the expected defect is found, 5 when an issue is flagged without matching that defect, and 0 when it is missed. Multi-defect video scores reflect the share of expected defects matched. Semantic interpretation resolves eligible answer matches. The pack does not establish a clean-ad false-alarm rate.
Creative tagging
Label the features of image and video ads using a fixed set of creative categories.
Highest published task score: 9.73/10 — Claude Fable 5.1. Provisional evidence.
Compare 17 models
Published model scores Model Score / setup Cost per scored output Claude Fable 5.1Provisional 9.73Combined setups $0.384API-equivalent estimate Meta Muse WonderProvisional 9.59Combined setups $0.063recorded spend Gemini 3.5 FlashProvisional 9.51Combined setups $0.004recorded spend Google Gemini 4 ArgonProvisional 9.49Combined setups $0.046recorded spend Gemini 3.8 FlashProvisional 9.47Combined setups $0.026recorded spend GPT-6 AstraProvisional 9.47Combined setups $0.075recorded spend Claude Opus 5.5Provisional 9.43Combined setups $0.339API-equivalent estimate Gemini 3.7 FlashProvisional 9.42Combined setups $0.004recorded spend Gemini 3.1 Pro PreviewProvisional 9.42Combined setups $0.027recorded spend Claude Opus 4.8Provisional 9.37Combined setups $0.045recorded spend Claude Opus 5Provisional 9.36Combined setups $0.061recorded spend GPT-6 SolProvisional 9.33Combined setups $0.017recorded spend GPT-5.5Provisional 9.25Combined setups $0.098recorded spend GPT-5.6 SolProvisional 9.24Combined setups $0.029recorded spend Claude Fable 5Provisional 9.21Combined setups $0.156API-equivalent estimate Meta Muse Spark 1.3Provisional 9.19Combined setups $0.019recorded spend Google: Gemma 4 26B A4BProvisional 8.97Combined setups Not recorded Inspect this task · Read the findings →
How we test this
Models label image and video ads using a defined taxonomy. The published score averages tag-field correctness per output; critical-field checks distinguish a mostly correct record from one suitable for downstream analysis.
What the model receives
The fixed media pack contains five images and five videos. Models return structured tags such as format, brand, product category, offer and CTA. Coverage varies across the published rows, so the results table retains each model's recorded output count.
What changes between prompts
Two prompt conditions are defined. Baseline receives allowed values and the JSON shape; the scaffold adds full definitions and supporting-evidence requirements for specified positive tags. Published model scores currently combine setups, with no complete paired comparison available on the site.
How the result is scored
Each output's score is its fraction of correctly resolved applicable fields, using exact checks and a semantic matcher where specified. The percentage shown here is the published mean 0–10 score multiplied by ten: mean correctness per output, not a pooled field-weighted accuracy or the percentage of entirely correct ads. The critical fields include format, brand, product category, offer and CTA; a high average can still contain a consequential error.
Asset naming and versioning
Propose clear file names and version numbers for a set of assets, spotting duplicates and clashes along the way.
Highest published task score: 8.65/10 — Claude Opus 5.5. Provisional evidence.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Claude Opus 5.5Provisional 8.65Scaffolded prompt Not recorded Meta Muse Spark 1.3Provisional 8.55Simple prompt Not recorded Claude Opus 5Provisional 8.51Scaffolded prompt Not recorded Claude Fable 5Provisional 8.33Scaffolded prompt $1.982API-equivalent estimate Gemini 3.7 FlashProvisional 8.26Scaffolded prompt Not recorded Meta Muse WonderProvisional 8.22Scaffolded prompt Not recorded Google Gemini 4 ArgonProvisional 8.21Simple prompt Not recorded GPT-6 SolProvisional 8.08Scaffolded prompt Not recorded GPT-6 AstraProvisional 8.08Simple prompt $0.752recorded spend Gemini 3.5 FlashProvisional 8.01Simple prompt Not recorded GPT-5.6 SolProvisional 7.89Simple prompt Not recorded Claude Opus 4.8Provisional 7.73Simple prompt Not recorded GPT-5.5Provisional 7.61Simple prompt Not recorded Gemini 3.6 FlashProvisional 7.60Scaffolded prompt Not recorded Claude Fable 5.1Provisional 6.48Simple prompt $2.217API-equivalent estimate Gemini 3.8 FlashProvisional 5.27Scaffolded prompt Not recorded Inspect this task · Read the findings →
How we test this
Models reconcile asset metadata, content hashes, a registry and naming policy into a proposed change set. Two synthetic packs contain 48 and 24 operations. The scorer checks exact names, versions, lineage, duplicates and collision handling.
What the model receives
The model receives stable asset IDs, existing filenames, content hashes, version relationships, requested operations and the full naming policy. Cases include duplicates under different names, conflicting versions, case/Unicode collisions and ambiguous parentage. The 72 operation rows belong to two packs; repeat outputs are not additional packs.
What changes between prompts
Both conditions receive the complete policy, facts and output schema. The scaffold adds an ordered process for identity reconciliation, naming, lineage, collision checks and escalation. The output is a dry-run proposal; the evaluation does not rename files.
How the result is scored
A deterministic exact-key scorer assesses filenames, actions, versions, parentage, duplicates, exceptions and idempotence. The mean correctness score is accompanied by strict whole-change-set and critical-safety checks. An attractive average does not mean every proposed operation is safe to apply; whole-set acceptance is the operational decision.
Deployment QA checklist
Check a campaign launch pack for missing information and mistakes, then say whether it is ready to go live.
Highest published task score: 10.00/10 — Meta Muse Spark 1.3, Gemini 3.7 Flash and 12 other models. Provisional evidence.
Equal top scores do not establish equivalent performance. Cost figures may use different bases or pool prompt setups.
Compare 15 models
Published model scores Model Score / setup Cost per scored output Meta Muse Spark 1.3Provisional 10.00Combined setups $0.099recorded spend Gemini 3.7 FlashProvisional 10.00Simple prompt $0.107recorded spend Gemini 3.8 FlashProvisional 10.00Combined setups $0.177recorded spend GPT-6 SolProvisional 10.00Combined setups $0.196recorded spend Google Gemini 4 ArgonProvisional 10.00Combined setups $0.245recorded spend Meta Muse WonderProvisional 10.00Combined setups $0.334recorded spend GPT-5.6 SolProvisional 10.00Simple prompt $0.516recorded spend GPT-5.5Provisional 10.00Simple prompt $0.589recorded spend Claude Opus 5Provisional 10.00Simple prompt $0.695recorded spend Claude Opus 5.5Provisional 10.00Combined setups $0.945API-equivalent estimate GPT-6 AstraProvisional 10.00Combined setups $0.999recorded spend Claude Fable 5Provisional 10.00Simple prompt $1.541API-equivalent estimate Claude Fable 5.1Provisional 10.00Combined setups $1.765API-equivalent estimate Claude Opus 4.8Provisional 10.00Simple prompt Not recorded Gemini 3.1 Pro PreviewProvisional 9.99Scaffolded prompt $0.191recorded spend How we test this
The V9 test asks models to check a synthetic launch package against supplied rules, point to the relevant evidence and select permitted repairs. It measures a structured launch check, with no live platform changes.
What the model receives
V9 supplies a frozen deployment pack, platform schema, complete rule registry, permitted repair actions and an exact response contract. Each pack has six bundles and nine checks per bundle. The verified original publication cohort contains seven models across four packs and three repeats per prompt condition.
What changes between prompts
Baseline and scaffolded conditions receive the same facts, rules, allowed actions and response contract. The scaffold adds an ordered inspection method. It does not grant new launch permissions or supply evaluator answers.
How the result is scored
The deterministic scorer checks rule decisions, evidence pointers, escalation and supplied repair selection. The original seven-model comparison covers four packs. Later published model rows have different coverage; compare their recorded sample counts alongside the score. A technical pass in this synthetic contract is a launch-check result, not permission to publish a campaign.
Performance readout
Read campaign performance data, explain what has changed and suggest what to do about it.
Highest published task score: 9.36/10 — Claude Fable 5.1. Provisional evidence.
Compare 18 models
Published model scores Model Score / setup Cost per scored output Claude Fable 5.1Provisional 9.36Simple prompt $2.251API-equivalent estimate Google Gemini 4 ArgonProvisional 9.14Scaffolded prompt $0.244recorded spend Claude Opus 5Provisional 9.00Scaffolded prompt $0.738recorded spend Claude Opus 5.5Provisional 8.86Simple prompt $2.041API-equivalent estimate Meta Muse Spark 1.3Provisional 8.64Scaffolded prompt $0.082recorded spend GPT-6 AstraProvisional 8.64Scaffolded prompt $0.977recorded spend GPT-6.1 SolProvisional 8.29Scaffolded prompt $0.180recorded spend Meta Muse WonderProvisional 8.25Scaffolded prompt $0.326recorded spend Gemini 3.8 FlashProvisional 8.07Scaffolded prompt $0.128recorded spend GPT-5.5Provisional 7.47Scaffolded prompt $0.277recorded spend GPT-6 SolProvisional 7.43Scaffolded prompt $0.150recorded spend Claude Fable 5Provisional 7.38Simple prompt $1.201API-equivalent estimate Gemini 3.7 FlashProvisional 7.12Scaffolded prompt $0.019recorded spend GPT-5.6 SolProvisional 6.81Scaffolded prompt $0.411recorded spend Gemini 3.5 FlashProvisional 6.62Combined setups $0.022recorded spend Gemini 3.6 FlashProvisional 6.61Combined setups $0.038recorded spend Claude Opus 4.8Provisional 6.57Combined setups $0.179recorded spend Gemini 3.1 Pro PreviewProvisional 6.00Combined setups $0.021recorded spend Inspect this task · Read the findings →
How we test this
Models analyse seven synthetic campaign datasets: five anchor cases and two holdouts, each with a known finding. Both prompts receive the data needed to identify it. Scores measure recovery of the expected analysis and action.
What the model receives
The supplied tables contain patterns such as a two-day placement spike, unusual daily spend and a shift towards post-view conversions. Seven datasets tested under two prompt conditions produce fourteen outputs per run.
What changes between prompts
The baseline is an analytical brief. The scaffold adds a generic diagnostic checklist covering aggregation, breakdowns, unreliable data and one-off effects. Neither identifies the planted finding.
How the result is scored
Weighted answer-key scoring and critical-miss caps measure whether the model recovers the expected finding, explains it and recommends a supported action. The dedicated article and reference preserve the full reviewed interpretation and permanent results.
Why an ad worked
Look at an ad and explain why it may have worked. Suggestions for the next version are scored separately.
Highest published task score: 9.93/10 — Google Gemini 4 Argon, Claude Opus 5. Provisional evidence.
Equal top scores do not establish equivalent performance. Cost figures may use different bases or pool prompt setups.
Compare 13 models
Published model scores Model Score / setup Cost per scored output Google Gemini 4 ArgonProvisional 9.93Scaffolded prompt $0.035recorded spend Claude Opus 5Provisional 9.93Scaffolded prompt $0.124recorded spend Claude Fable 5.1Provisional 9.90Scaffolded prompt $0.415API-equivalent estimate Claude Opus 5.5Provisional 9.90Scaffolded prompt $0.622API-equivalent estimate Gemini 3.7 FlashProvisional 9.73Scaffolded prompt $0.005recorded spend Gemini 3.8 FlashProvisional 9.70Scaffolded prompt $0.010recorded spend Meta Muse Spark 1.3Provisional 9.67Scaffolded prompt $0.018recorded spend Meta Muse WonderProvisional 9.43Scaffolded prompt $0.081recorded spend Gemini 3.5 FlashProvisional 9.42Scaffolded prompt $0.006recorded spend GPT-6 AstraProvisional 9.33Scaffolded prompt $0.154recorded spend Gemini 3.1 Pro PreviewProvisional 9.32Scaffolded prompt $0.008recorded spend Gemini 3.1 Flash-LiteProvisional 8.94Scaffolded prompt $0.001recorded spend GPT-6 SolProvisional 8.00Scaffolded prompt $0.022recorded spend Inspect this task · Read the findings →
How we test this
Models inspect five image ads and five video ads and explain their creative mechanisms. Ben blind-scores diagnostic accuracy, depth and explanation quality. Performance data is withheld, so the task measures creative diagnosis rather than causal attribution of sales.
What the model receives
The fixed pack contains five static images and five videos. Models see the media, with original filenames and account performance withheld. They receive no spend, conversion, reach or brand-context evidence. Analysis and creative recommendations are generated as separate outputs.
What changes between prompts
Each asset is tested with a simple and a scaffolded prompt. The analysis scaffold covers attention, product relevance, proof, platform fit and other creative mechanisms. The action scaffold asks what to preserve, what to change and why. The media remains identical between conditions.
How the result is scored
The explanation is scored on diagnostic accuracy, depth of analysis and quality of explanation. Diagnostic accuracy is a critical dimension: a persuasive explanation with a weak diagnosis does not clear the task's readiness threshold. Recommendations for the next ad have their own score and page.
Creative diversity and coverage
Look across an ad portfolio to show what is repeated, what is missing and where a new creative test could help.
Highest published task score: 10.00/10 — Gemini 3.8 Flash, Gemini 3.7 Flash and 14 other models. Provisional evidence.
Equal top scores do not establish equivalent performance. Cost figures may use different bases or pool prompt setups.
Compare 16 models
Published model scores Model Score / setup Cost per scored output Gemini 3.8 FlashProvisional 10.00Scaffolded prompt $0.507recorded spend Gemini 3.7 FlashProvisional 10.00Simple prompt Not recorded Meta Muse Spark 1.3Provisional 10.00Simple prompt $0.557recorded spend Gemini 3.5 FlashProvisional 10.00Scaffolded prompt Not recorded Gemini 3.1 Pro PreviewProvisional 10.00Scaffolded prompt Not recorded GPT-6 SolProvisional 10.00Simple prompt $1.674recorded spend Google Gemini 4 ArgonProvisional 10.00Scaffolded prompt $1.928recorded spend GPT-5.5Provisional 10.00Simple prompt Not recorded GPT-6 AstraProvisional 10.00Scaffolded prompt $3.129recorded spend Claude Opus 4.8Provisional 10.00Simple prompt Not recorded Claude Opus 5Provisional 10.00Simple prompt Not recorded GPT-5.6 SolProvisional 10.00Simple prompt Not recorded Meta Muse WonderProvisional 10.00Scaffolded prompt $4.089recorded spend Claude Fable 5.1Provisional 10.00Simple prompt Not recorded Claude Fable 5Provisional 10.00Simple prompt Not recorded Claude Opus 5.5Provisional 10.00Simple prompt Not recorded Inspect this task · Read the findings →
How we test this
Models analyse the same fixed account's reviewed creative labels and identify concentrations, gaps and useful next tests. The benchmark compares the analysis layer; it does not combine tagging accuracy with the headline analysis score.
What the model receives
The account pack contains 35 files, including one exact duplicate, representing 34 unique assets. Both analysis conditions receive the fixed reviewed taxonomy records. One account is one case even when the model inspects many asset rows.
What changes between prompts
The simple and scaffolded analysis prompts use the same labelled portfolio. The scaffold structures the concentration, risk, gap and recommendation assessment. The earlier classification/instrument workflow is separate from this comparison.
How the result is scored
The evaluation checks analysis against an answer key, including numerical consistency, real risks, unsupported risks, gaps and deliberately absent categories. Recommendation quality and operator readiness have a separate judged layer. Read the published score as performance on this fixed analysis task, rather than proof that an arbitrary portfolio can be classified and planned end to end.
Insight codification
Turn campaign results into clear, reusable learnings, and update them when new evidence comes in.
Highest published task score: 8.25/10 — Gemini 3.5 Flash. Provisional evidence.
Compare 17 models
Published model scores Model Score / setup Cost per scored output Gemini 3.5 FlashProvisional 8.25Scaffolded prompt $0.011recorded spend Meta Muse WonderProvisional 7.71Scaffolded prompt $0.386recorded spend Gemini 3.7 FlashProvisional 7.57Scaffolded prompt $0.009recorded spend Claude Opus 5.5Provisional 7.56Scaffolded prompt Not recorded Meta Muse Spark 1.3Provisional 7.44Scaffolded prompt $0.093recorded spend Google Gemini 4 ArgonProvisional 7.44Scaffolded prompt $0.282recorded spend Claude Fable 5Provisional 7.44Scaffolded prompt $1.080API-equivalent estimate Gemini 3.6 FlashProvisional 6.99Combined setups $0.021recorded spend Gemini 3.1 Pro PreviewProvisional 6.76Combined setups Not recorded GPT-6 SolProvisional 6.65Simple prompt $0.098recorded spend Claude Opus 5Provisional 6.51Combined setups $0.498recorded spend GPT-6 AstraProvisional 6.50Scaffolded prompt $0.595recorded spend GPT-5.6 SolProvisional 5.77Combined setups $0.219recorded spend GPT-5.5Provisional 5.73Scaffolded prompt $0.037recorded spend Claude Opus 4.8Provisional 5.64Combined setups $0.243recorded spend Claude Fable 5.1Provisional 5.62Scaffolded prompt Not recorded Gemini 3.8 FlashProvisional 5.25Scaffolded prompt $0.061recorded spend Inspect this task · Read the findings →
How we test this
Models extract reusable learning from synthetic campaign results and update an existing insight library when new evidence arrives. The scorer checks genuine signals, misleading patterns, confidence and the correct memory-update decisions.
What the model receives
The signal-extraction pack contains 22 ads with known signals, decoys, low-evidence traps and a data-quality issue. The memory-update pack contains six existing insights and a ten-ad follow-up digest with eight decision points. These are two controlled cases; the model rows have different coverage and repeat counts.
What changes between prompts
The baseline states the task and output shape. The scaffold adds evidence floors, a confound checklist, calibration rules and an update vocabulary. Both use the same campaign evidence; the evaluator-only answers are withheld.
How the result is scored
The deterministic composite checks signal recovery, false insights, confidence, traps and memory updates against the case key. It rewards retaining, narrowing or replacing a learning when the evidence warrants it. It does not score fluent summary writing or measure a downstream campaign lift.
Which model fits the task?
Models have different task coverage. These comparisons retain each task’s own rubric; a broad average is not a recommendation for every job.
Claude Fable 5 · 14 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.43Simple prompt | Provisional |
| Brand context to concepts | 6.00Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 6.11Simple prompt | Provisional |
| Idea scoring and priority plan | 9.83Scaffolded prompt | Provisional |
| Hook development | 8.00Scaffolded prompt | Published |
| Narrative development | 8.00Simple prompt | Published |
| Compliant brief generation | 9.50Simple prompt | Published |
| Content review | 5.50Scaffolded prompt | Provisional |
| Creative tagging | 9.21Combined setups | Provisional |
| Asset naming and versioning | 8.33Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Simple prompt | Provisional |
| Performance readout | 7.38Simple prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 7.44Scaffolded prompt | Provisional |
Claude Fable 5.1 · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.50Scaffolded prompt | Provisional |
| Brand context to concepts | 6.67Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 7.00Simple prompt | Provisional |
| Iterate on a top performer | 8.83Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 10.00Simple prompt | Provisional |
| Hook development | 6.75Scaffolded prompt | Published |
| Narrative development | 8.00Scaffolded prompt | Published |
| Compliant brief generation | 9.50Simple prompt | Published |
| Content review | 6.38Simple prompt | Provisional |
| Creative tagging | 9.73Combined setups | Provisional |
| Asset naming and versioning | 6.48Simple prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 9.36Simple prompt | Provisional |
| Why an ad worked | 9.90Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 5.62Scaffolded prompt | Provisional |
Claude Opus 4.7 · 2 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Hook development | 8.00Scaffolded prompt | Provisional |
| Narrative development | 8.58Simple prompt | Provisional |
Claude Opus 4.8 · 14 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.26Simple prompt | Provisional |
| Brand context to concepts | 6.59Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.74Simple prompt | Provisional |
| Idea scoring and priority plan | 10.00Simple prompt | Provisional |
| Hook development | 7.83Scaffolded prompt | Provisional |
| Narrative development | 8.17Simple prompt | Provisional |
| Compliant brief generation | 9.39Scaffolded prompt | Provisional |
| Content review | 5.12Scaffolded prompt | Provisional |
| Creative tagging | 9.37Combined setups | Provisional |
| Asset naming and versioning | 7.73Simple prompt | Provisional |
| Deployment QA checklist | 10.00Simple prompt | Provisional |
| Performance readout | 6.57Combined setups | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 5.64Combined setups | Provisional |
Claude Opus 5 · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 6.92Simple prompt | Provisional |
| Brand context to concepts | 5.78Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.78Simple prompt | Provisional |
| Iterate on a top performer | 9.47Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 9.92Combined setups | Provisional |
| Hook development | 8.50Scaffolded prompt | Published |
| Narrative development | 8.50Scaffolded prompt | Published |
| Compliant brief generation | 9.50Scaffolded prompt | Published |
| Content review | 6.24Simple prompt | Provisional |
| Creative tagging | 9.36Combined setups | Provisional |
| Asset naming and versioning | 8.51Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Simple prompt | Provisional |
| Performance readout | 9.00Scaffolded prompt | Provisional |
| Why an ad worked | 9.93Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 6.51Combined setups | Provisional |
Claude Opus 5.5 · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.72Simple prompt | Provisional |
| Brand context to concepts | 8.11Simple prompt | Provisional |
| Insights and ads to a new proposal | 7.67Simple prompt | Provisional |
| Iterate on a top performer | 9.43Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 10.00Scaffolded prompt | Provisional |
| Hook development | 6.75Simple prompt | Published |
| Narrative development | 7.75Simple prompt | Published |
| Compliant brief generation | 9.50Simple prompt | Published |
| Content review | 7.79Simple prompt | Provisional |
| Creative tagging | 9.43Combined setups | Provisional |
| Asset naming and versioning | 8.65Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 8.86Simple prompt | Provisional |
| Why an ad worked | 9.90Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 7.56Scaffolded prompt | Provisional |
Gemini 3 Pro Image · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Static image generation | 7.12Simple prompt | Provisional |
Gemini 3.1 Flash-Lite · 4 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Iterate on a top performer | 8.23Scaffolded prompt | Provisional |
| Hook development | 7.83Simple prompt | Provisional |
| Narrative development | 7.75Scaffolded prompt | Provisional |
| Why an ad worked | 8.94Scaffolded prompt | Provisional |
Gemini 3.1 Pro Preview · 15 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.89Simple prompt | Provisional |
| Brand context to concepts | 7.33Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 7.22Scaffolded prompt | Provisional |
| Iterate on a top performer | 8.78Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 9.92Scaffolded prompt | Provisional |
| Hook development | 8.00Scaffolded prompt | Provisional |
| Narrative development | 7.83Simple prompt | Provisional |
| Compliant brief generation | 9.37Scaffolded prompt | Published |
| Content review | 6.79Scaffolded prompt | Provisional |
| Creative tagging | 9.42Combined setups | Provisional |
| Deployment QA checklist | 9.99Scaffolded prompt | Provisional |
| Performance readout | 6.00Combined setups | Provisional |
| Why an ad worked | 9.32Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Scaffolded prompt | Provisional |
| Insight codification | 6.76Combined setups | Provisional |
Gemini 3.5 Flash · 15 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.56Simple prompt | Provisional |
| Brand context to concepts | 6.56Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 6.85Simple prompt | Provisional |
| Iterate on a top performer | 9.01Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 9.36Simple prompt | Provisional |
| Hook development | 6.29Scaffolded prompt | Published |
| Narrative development | 6.86Scaffolded prompt | Published |
| Compliant brief generation | 8.81Simple prompt | Published |
| Content review | 8.38Scaffolded prompt | Provisional |
| Creative tagging | 9.51Combined setups | Provisional |
| Asset naming and versioning | 8.01Simple prompt | Provisional |
| Performance readout | 6.62Combined setups | Provisional |
| Why an ad worked | 9.42Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Scaffolded prompt | Provisional |
| Insight codification | 8.25Scaffolded prompt | Provisional |
Gemini 3.6 Flash · 4 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Idea scoring and priority plan | 9.75Combined setups | Provisional |
| Asset naming and versioning | 7.60Scaffolded prompt | Provisional |
| Performance readout | 6.61Combined setups | Provisional |
| Insight codification | 6.99Combined setups | Provisional |
Gemini 3.7 Flash · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.30Simple prompt | Provisional |
| Brand context to concepts | 6.63Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 6.74Simple prompt | Provisional |
| Iterate on a top performer | 9.22Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 10.00Scaffolded prompt | Provisional |
| Hook development | 8.50Scaffolded prompt | Published |
| Narrative development | 7.75Simple prompt | Published |
| Compliant brief generation | 9.17Scaffolded prompt | Published |
| Content review | 6.68Scaffolded prompt | Provisional |
| Creative tagging | 9.42Combined setups | Provisional |
| Asset naming and versioning | 8.26Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Simple prompt | Provisional |
| Performance readout | 7.12Scaffolded prompt | Provisional |
| Why an ad worked | 9.73Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 7.57Scaffolded prompt | Provisional |
Gemini 3.8 Flash · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.89Simple prompt | Provisional |
| Brand context to concepts | 6.33Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.89Simple prompt | Provisional |
| Iterate on a top performer | 8.87Simple prompt | Provisional |
| Idea scoring and priority plan | 10.00Scaffolded prompt | Provisional |
| Hook development | 7.00Simple prompt | Published |
| Narrative development | 7.75Simple prompt | Published |
| Compliant brief generation | 9.50Scaffolded prompt | Published |
| Content review | 6.25Scaffolded prompt | Provisional |
| Creative tagging | 9.47Combined setups | Provisional |
| Asset naming and versioning | 5.27Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 8.07Scaffolded prompt | Provisional |
| Why an ad worked | 9.70Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Scaffolded prompt | Provisional |
| Insight codification | 5.25Scaffolded prompt | Provisional |
Gemini Omni Flash · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Video generation | 6.68Scaffolded prompt | Provisional |
Gen-4.5 · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Video generation | 5.24Scaffolded prompt | Provisional |
Google Gemini 4 Argon · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 8.33Simple prompt | Provisional |
| Brand context to concepts | 7.22Simple prompt | Provisional |
| Insights and ads to a new proposal | 7.33Simple prompt | Provisional |
| Iterate on a top performer | 9.33Simple prompt | Provisional |
| Idea scoring and priority plan | 10.00Simple prompt | Provisional |
| Hook development | 8.25Scaffolded prompt | Published |
| Narrative development | 7.75Simple prompt | Published |
| Compliant brief generation | 9.17Simple prompt | Published |
| Content review | 8.00Simple prompt | Provisional |
| Creative tagging | 9.49Combined setups | Provisional |
| Asset naming and versioning | 8.21Simple prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 9.14Scaffolded prompt | Provisional |
| Why an ad worked | 9.93Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Scaffolded prompt | Provisional |
| Insight codification | 7.44Scaffolded prompt | Provisional |
Google: Gemma 4 26B A4B · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Creative tagging | 8.97Combined setups | Provisional |
GPT Image 2 · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Static image generation | 7.44Scaffolded prompt | Provisional |
GPT Image 2.5 Sunburst · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Static image generation | 8.84Scaffolded prompt | Provisional |
GPT-5.4 Mini · 2 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Hook development | 7.58Scaffolded prompt | Provisional |
| Narrative development | 7.75Simple prompt | Provisional |
GPT-5.5 · 14 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.41Simple prompt | Provisional |
| Brand context to concepts | 6.30Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.67Simple prompt | Provisional |
| Idea scoring and priority plan | 9.83Combined setups | Provisional |
| Hook development | 7.58Simple prompt | Provisional |
| Narrative development | 8.00Simple prompt | Provisional |
| Compliant brief generation | 9.38Scaffolded prompt | Published |
| Content review | 5.18Scaffolded prompt | Provisional |
| Creative tagging | 9.25Combined setups | Provisional |
| Asset naming and versioning | 7.61Simple prompt | Provisional |
| Deployment QA checklist | 10.00Simple prompt | Provisional |
| Performance readout | 7.47Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 5.73Scaffolded prompt | Provisional |
GPT-5.6 Sol · 14 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.11Simple prompt | Provisional |
| Brand context to concepts | 6.33Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 6.41Simple prompt | Provisional |
| Idea scoring and priority plan | 9.75Combined setups | Provisional |
| Hook development | 7.75Simple prompt | Published |
| Narrative development | 8.00Scaffolded prompt | Published |
| Compliant brief generation | 9.50Scaffolded prompt | Published |
| Content review | 5.22Scaffolded prompt | Provisional |
| Creative tagging | 9.24Combined setups | Provisional |
| Asset naming and versioning | 7.89Simple prompt | Provisional |
| Deployment QA checklist | 10.00Simple prompt | Provisional |
| Performance readout | 6.81Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 5.77Combined setups | Provisional |
GPT-6 Astra · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.28Simple prompt | Provisional |
| Brand context to concepts | 6.11Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.22Scaffolded prompt | Provisional |
| Iterate on a top performer | 8.03Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 10.00Simple prompt | Provisional |
| Hook development | 8.00Scaffolded prompt | Published |
| Narrative development | 7.50Scaffolded prompt | Published |
| Compliant brief generation | 9.50Scaffolded prompt | Published |
| Content review | 5.12Scaffolded prompt | Provisional |
| Creative tagging | 9.47Combined setups | Provisional |
| Asset naming and versioning | 8.08Simple prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 8.64Scaffolded prompt | Provisional |
| Why an ad worked | 9.33Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Scaffolded prompt | Provisional |
| Insight codification | 6.50Scaffolded prompt | Provisional |
GPT-6 Sol · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.89Simple prompt | Provisional |
| Brand context to concepts | 6.78Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.56Simple prompt | Provisional |
| Iterate on a top performer | 8.23Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 10.00Simple prompt | Provisional |
| Hook development | 6.75Simple prompt | Published |
| Narrative development | 7.50Simple prompt | Published |
| Compliant brief generation | 9.50Simple prompt | Published |
| Content review | 4.96Scaffolded prompt | Provisional |
| Creative tagging | 9.33Combined setups | Provisional |
| Asset naming and versioning | 8.08Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 7.43Scaffolded prompt | Provisional |
| Why an ad worked | 8.00Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 6.65Simple prompt | Provisional |
GPT-6.1 Sol · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Performance readout | 8.29Scaffolded prompt | Provisional |
Meta Muse Spark 1.3 · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.33Simple prompt | Provisional |
| Brand context to concepts | 6.84Scaffolded prompt | Provisional |
| Insights and ads to a new proposal | 7.00Scaffolded prompt | Provisional |
| Iterate on a top performer | 8.70Scaffolded prompt | Provisional |
| Idea scoring and priority plan | 10.00Scaffolded prompt | Provisional |
| Hook development | 8.75Scaffolded prompt | Published |
| Narrative development | 7.50Scaffolded prompt | Published |
| Compliant brief generation | 9.50Scaffolded prompt | Published |
| Content review | 5.38Simple prompt | Provisional |
| Creative tagging | 9.19Combined setups | Provisional |
| Asset naming and versioning | 8.55Simple prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 8.64Scaffolded prompt | Provisional |
| Why an ad worked | 9.67Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Simple prompt | Provisional |
| Insight codification | 7.44Scaffolded prompt | Provisional |
Meta Muse Wonder · 16 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Reference ad to new concept | 7.39Simple prompt | Provisional |
| Brand context to concepts | 6.56Simple prompt | Provisional |
| Insights and ads to a new proposal | 6.56Simple prompt | Provisional |
| Iterate on a top performer | 9.03Simple prompt | Provisional |
| Idea scoring and priority plan | 10.00Simple prompt | Provisional |
| Hook development | 7.50Simple prompt | Published |
| Narrative development | 6.50Scaffolded prompt | Published |
| Compliant brief generation | 9.50Simple prompt | Published |
| Content review | 6.83Simple prompt | Provisional |
| Creative tagging | 9.59Combined setups | Provisional |
| Asset naming and versioning | 8.22Scaffolded prompt | Provisional |
| Deployment QA checklist | 10.00Combined setups | Provisional |
| Performance readout | 8.25Scaffolded prompt | Provisional |
| Why an ad worked | 9.43Scaffolded prompt | Provisional |
| Creative diversity and coverage | 10.00Scaffolded prompt | Provisional |
| Insight codification | 7.71Scaffolded prompt | Provisional |
Seedance 2 · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Video generation | 4.56Scaffolded prompt | Provisional |
Seedream 5 Pro · 1 scored tasks
| Task | Score / setup | Evidence |
|---|---|---|
| Static image generation | 7.64Simple prompt | Provisional |
Does extra guidance help?
These complete paired comparisons show both prompt setups, including lower and unchanged scaffold scores. Repeated outputs can reuse datasets; small differences may not generalise.
Reference ad to new concept · 12 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 7.43 | 6.72 | 18 |
| Claude Fable 5.1Provisional | 7.39 | 7.50 | 6 |
| Claude Opus 4.8Provisional | 7.26 | 6.91 | 18 |
| Claude Opus 5.5Provisional | 7.72 | 6.94 | 6 |
| Gemini 3.5 FlashProvisional | 7.56 | 7.54 | 18 |
| Gemini 3.7 FlashProvisional | 7.30 | 7.13 | 18 |
| Gemini 3.8 FlashProvisional | 7.89 | 7.28 | 6 |
| GPT-5.5Provisional | 7.41 | 6.78 | 18 |
| GPT-5.6 SolProvisional | 7.11 | 6.19 | 18 |
| GPT-6 AstraProvisional | 7.28 | 5.50 | 6 |
| GPT-6 SolProvisional | 7.89 | 5.78 | 6 |
| Google Gemini 4 ArgonProvisional | 8.33 | 7.72 | 6 |
Brand context to concepts · 14 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 5.96 | 6.00 | 9 |
| Claude Fable 5.1Provisional | 6.56 | 6.67 | 3 |
| Claude Opus 4.8Provisional | 6.59 | 6.15 | 9 |
| Claude Opus 5.5Provisional | 8.11 | 8.11 | 3 |
| Gemini 3.5 FlashProvisional | 6.11 | 6.56 | 9 |
| Gemini 3.7 FlashProvisional | 6.30 | 6.63 | 9 |
| Gemini 3.8 FlashProvisional | 6.33 | 6.11 | 3 |
| GPT-5.5Provisional | 6.30 | 6.18 | 9 |
| GPT-5.6 SolProvisional | 5.59 | 6.33 | 9 |
| GPT-6 AstraProvisional | 6.11 | 5.89 | 3 |
| GPT-6 SolProvisional | 6.78 | 6.11 | 3 |
| Google Gemini 4 ArgonProvisional | 7.22 | 7.11 | 3 |
| Meta Muse Spark 1.3Provisional | 6.44 | 6.84 | 15 |
| Meta Muse WonderProvisional | 6.56 | 6.44 | 3 |
Insights and ads to a new proposal · 14 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 6.11 | 6.04 | 9 |
| Claude Fable 5.1Provisional | 7.00 | 6.78 | 3 |
| Claude Opus 4.8Provisional | 6.74 | 6.37 | 9 |
| Claude Opus 5.5Provisional | 7.67 | 7.22 | 3 |
| Gemini 3.5 FlashProvisional | 6.85 | 6.44 | 9 |
| Gemini 3.7 FlashProvisional | 6.74 | 6.59 | 9 |
| Gemini 3.8 FlashProvisional | 6.89 | 6.11 | 3 |
| GPT-5.5Provisional | 6.67 | 6.56 | 9 |
| GPT-5.6 SolProvisional | 6.41 | 6.00 | 9 |
| GPT-6 AstraProvisional | 5.33 | 6.22 | 3 |
| GPT-6 SolProvisional | 6.56 | 6.56 | 3 |
| Google Gemini 4 ArgonProvisional | 7.33 | 6.89 | 3 |
| Meta Muse Spark 1.3Provisional | 6.56 | 7.00 | 3 |
| Meta Muse WonderProvisional | 6.56 | 6.56 | 3 |
Iterate on a top performer · 10 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5.1Provisional | 8.80 | 8.83 | 10 |
| Claude Opus 5.5Provisional | 9.20 | 9.43 | 10 |
| Gemini 3.1 Flash-LiteProvisional | 7.83 | 8.23 | 30 |
| Gemini 3.5 FlashProvisional | 8.72 | 9.01 | 30 |
| Gemini 3.7 FlashProvisional | 9.13 | 9.22 | 30 |
| Gemini 3.8 FlashProvisional | 8.87 | 8.83 | 10 |
| GPT-6 AstraProvisional | 8.00 | 8.03 | 10 |
| GPT-6 SolProvisional | 7.93 | 8.23 | 10 |
| Google Gemini 4 ArgonProvisional | 9.33 | 9.20 | 10 |
| Meta Muse Spark 1.3Provisional | 8.57 | 8.70 | 10 |
Idea scoring and priority plan · 11 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 9.33 | 9.83 | 3 |
| Claude Fable 5.1Provisional | 10.00 | 10.00 | 1 |
| Claude Opus 5.5Provisional | 9.50 | 10.00 | 1 |
| Gemini 3.1 Pro PreviewProvisional | 9.17 | 9.92 | 3 |
| Gemini 3.7 FlashProvisional | 9.92 | 10.00 | 3 |
| Gemini 3.8 FlashProvisional | 9.50 | 10.00 | 1 |
| GPT-6 AstraProvisional | 10.00 | 10.00 | 1 |
| GPT-6 SolProvisional | 10.00 | 10.00 | 1 |
| Google Gemini 4 ArgonProvisional | 10.00 | 10.00 | 1 |
| Meta Muse Spark 1.3Provisional | 9.50 | 10.00 | 1 |
| Meta Muse WonderProvisional | 10.00 | 10.00 | 1 |
Hook development · 19 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Published | 7.50 | 8.00 | 1 |
| Claude Fable 5.1Published | 6.50 | 6.75 | 1 |
| Claude Opus 4.7Provisional | 7.83 | 8.00 | 3 |
| Claude Opus 4.8Provisional | 7.08 | 7.83 | 3 |
| Claude Opus 5Published | 8.00 | 8.50 | 1 |
| Claude Opus 5.5Published | 6.75 | 6.50 | 1 |
| Gemini 3.1 Flash-LiteProvisional | 7.83 | 7.75 | 3 |
| Gemini 3.1 Pro PreviewProvisional | 7.75 | 8.00 | 3 |
| Gemini 3.5 FlashPublished | 5.89 | 6.29 | 7 |
| Gemini 3.7 FlashPublished | 7.75 | 8.50 | 1 |
| Gemini 3.8 FlashPublished | 7.00 | 7.00 | 1 |
| GPT-5.4 MiniProvisional | 7.50 | 7.58 | 3 |
| GPT-5.5Provisional | 7.58 | 7.50 | 3 |
| GPT-5.6 SolPublished | 7.75 | 7.50 | 1 |
| GPT-6 AstraPublished | 7.50 | 8.00 | 1 |
| GPT-6 SolPublished | 6.75 | 6.75 | 1 |
| Google Gemini 4 ArgonPublished | 7.75 | 8.25 | 1 |
| Meta Muse Spark 1.3Published | 7.50 | 8.75 | 1 |
| Meta Muse WonderPublished | 7.50 | 7.25 | 1 |
Narrative development · 19 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Published | 8.00 | 8.00 | 1 |
| Claude Fable 5.1Published | 7.50 | 8.00 | 1 |
| Claude Opus 4.7Provisional | 8.58 | 8.00 | 3 |
| Claude Opus 4.8Provisional | 8.17 | 8.08 | 3 |
| Claude Opus 5Published | 8.00 | 8.50 | 1 |
| Claude Opus 5.5Published | 7.75 | 7.75 | 1 |
| Gemini 3.1 Flash-LiteProvisional | 7.67 | 7.75 | 3 |
| Gemini 3.1 Pro PreviewProvisional | 7.83 | 7.67 | 3 |
| Gemini 3.5 FlashPublished | 6.32 | 6.86 | 7 |
| Gemini 3.7 FlashPublished | 7.75 | 7.50 | 1 |
| Gemini 3.8 FlashPublished | 7.75 | 7.00 | 1 |
| GPT-5.4 MiniProvisional | 7.75 | 7.25 | 3 |
| GPT-5.5Provisional | 8.00 | 7.83 | 3 |
| GPT-5.6 SolPublished | 7.50 | 8.00 | 1 |
| GPT-6 AstraPublished | 7.25 | 7.50 | 1 |
| GPT-6 SolPublished | 7.50 | 7.50 | 1 |
| Google Gemini 4 ArgonPublished | 7.75 | 7.75 | 1 |
| Meta Muse Spark 1.3Published | 6.75 | 7.50 | 1 |
| Meta Muse WonderPublished | 6.25 | 6.50 | 1 |
Compliant brief generation · 16 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Published | 9.50 | 9.50 | 1 |
| Claude Fable 5.1Published | 9.50 | 9.50 | 1 |
| Claude Opus 4.8Provisional | 9.17 | 9.39 | 6 |
| Claude Opus 5Published | 6.83 | 9.50 | 1 |
| Claude Opus 5.5Published | 9.50 | 9.50 | 1 |
| Gemini 3.1 Pro PreviewPublished | 9.10 | 9.37 | 5 |
| Gemini 3.5 FlashPublished | 8.81 | 8.67 | 7 |
| Gemini 3.7 FlashPublished | 8.50 | 9.17 | 1 |
| Gemini 3.8 FlashPublished | 8.50 | 9.50 | 1 |
| GPT-5.5Published | 9.35 | 9.38 | 10 |
| GPT-5.6 SolPublished | 9.17 | 9.50 | 1 |
| GPT-6 AstraPublished | 9.17 | 9.50 | 1 |
| GPT-6 SolPublished | 9.50 | 9.50 | 1 |
| Google Gemini 4 ArgonPublished | 9.17 | 8.00 | 1 |
| Meta Muse Spark 1.3Published | 9.17 | 9.50 | 1 |
| Meta Muse WonderPublished | 9.50 | 9.50 | 1 |
Static image generation · 3 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| GPT Image 2Provisional | 7.16 | 7.44 | 5 |
| GPT Image 2.5 SunburstPublished | 8.44 | 8.84 | 5 |
| Seedream 5 ProProvisional | 7.64 | 7.48 | 5 |
Video generation · 3 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Gemini Omni FlashProvisional | 6.00 | 6.68 | 5 |
| Gen-4.5Provisional | 4.52 | 5.24 | 5 |
| Seedance 2Provisional | 4.40 | 4.56 | 5 |
Content review · 16 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 5.08 | 5.50 | 20 |
| Claude Fable 5.1Provisional | 6.38 | 5.54 | 20 |
| Claude Opus 4.8Provisional | 4.88 | 5.13 | 20 |
| Claude Opus 5Provisional | 6.24 | 5.89 | 60 |
| Claude Opus 5.5Provisional | 7.79 | 7.79 | 20 |
| Gemini 3.1 Pro PreviewProvisional | 6.66 | 6.79 | 80 |
| Gemini 3.5 FlashProvisional | 8.00 | 8.38 | 40 |
| Gemini 3.7 FlashProvisional | 6.63 | 6.68 | 60 |
| Gemini 3.8 FlashProvisional | 5.96 | 6.25 | 20 |
| GPT-5.5Provisional | 4.69 | 5.18 | 80 |
| GPT-5.6 SolProvisional | 5.00 | 5.22 | 60 |
| GPT-6 AstraProvisional | 5.08 | 5.12 | 20 |
| GPT-6 SolProvisional | 4.83 | 4.96 | 20 |
| Google Gemini 4 ArgonProvisional | 8.00 | 7.92 | 20 |
| Meta Muse Spark 1.3Provisional | 5.38 | 5.38 | 20 |
| Meta Muse WonderProvisional | 6.83 | 6.46 | 20 |
Asset naming and versioning · 13 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 8.21 | 8.33 | 1 |
| Claude Fable 5.1Provisional | 6.48 | 5.60 | 6 |
| Claude Opus 4.8Provisional | 7.73 | 7.60 | 6 |
| Gemini 3.5 FlashProvisional | 8.01 | 7.94 | 6 |
| Gemini 3.6 FlashProvisional | 6.37 | 7.60 | 6 |
| Gemini 3.7 FlashProvisional | 6.82 | 8.26 | 6 |
| Gemini 3.8 FlashProvisional | 2.72 | 5.27 | 6 |
| GPT-5.6 SolProvisional | 7.89 | 7.89 | 6 |
| GPT-6 AstraProvisional | 8.08 | 7.75 | 1 |
| GPT-6 SolProvisional | 7.82 | 8.08 | 6 |
| Google Gemini 4 ArgonProvisional | 8.21 | 8.21 | 6 |
| Meta Muse Spark 1.3Provisional | 8.55 | 8.48 | 6 |
| Meta Muse WonderProvisional | 8.16 | 8.22 | 6 |
Deployment QA checklist · 7 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Published | 10.00 | 10.00 | 12 |
| Claude Opus 4.8Published | 10.00 | 10.00 | 12 |
| Claude Opus 5Published | 10.00 | 10.00 | 12 |
| Gemini 3.1 Pro PreviewPublished | 9.99 | 9.99 | 12 |
| Gemini 3.7 FlashPublished | 10.00 | 10.00 | 12 |
| GPT-5.5Published | 10.00 | 10.00 | 12 |
| GPT-5.6 SolPublished | 10.00 | 10.00 | 12 |
Performance readout · 10 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 7.38 | 6.69 | 21 |
| Claude Fable 5.1Provisional | 9.36 | 9.14 | 7 |
| Claude Opus 5.5Provisional | 8.86 | 8.07 | 7 |
| Gemini 3.7 FlashProvisional | 6.50 | 7.12 | 21 |
| Gemini 3.8 FlashProvisional | 7.86 | 8.07 | 7 |
| GPT-6 AstraProvisional | 7.50 | 8.64 | 7 |
| GPT-6 SolProvisional | 7.14 | 7.43 | 7 |
| GPT-6.1 SolProvisional | 8.07 | 8.29 | 7 |
| Google Gemini 4 ArgonProvisional | 8.36 | 9.14 | 7 |
| Meta Muse Spark 1.3Provisional | 7.36 | 8.64 | 7 |
Why an ad worked · 10 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5.1Provisional | 9.67 | 9.90 | 10 |
| Claude Opus 5.5Provisional | 9.77 | 9.90 | 10 |
| Gemini 3.1 Flash-LiteProvisional | 8.33 | 8.94 | 30 |
| Gemini 3.5 FlashProvisional | 9.03 | 9.42 | 30 |
| Gemini 3.7 FlashProvisional | 9.07 | 9.73 | 30 |
| Gemini 3.8 FlashProvisional | 9.17 | 9.70 | 10 |
| GPT-6 AstraProvisional | 8.67 | 9.33 | 10 |
| GPT-6 SolProvisional | 7.50 | 8.00 | 10 |
| Google Gemini 4 ArgonProvisional | 9.30 | 9.93 | 10 |
| Meta Muse Spark 1.3Provisional | 8.83 | 9.67 | 10 |
Creative diversity and coverage · 16 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Published | 10.00 | 10.00 | 2 |
| Claude Fable 5.1Provisional | 10.00 | 10.00 | 2 |
| Claude Opus 4.8Published | 10.00 | 10.00 | 2 |
| Claude Opus 5Published | 10.00 | 10.00 | 2 |
| Claude Opus 5.5Provisional | 10.00 | 10.00 | 2 |
| Gemini 3.1 Pro PreviewPublished | 8.34 | 10.00 | 2 |
| Gemini 3.5 FlashPublished | 6.67 | 10.00 | 2 |
| Gemini 3.7 FlashPublished | 10.00 | 10.00 | 2 |
| Gemini 3.8 FlashProvisional | 8.34 | 10.00 | 2 |
| GPT-5.5Published | 10.00 | 10.00 | 2 |
| GPT-5.6 SolPublished | 10.00 | 10.00 | 2 |
| GPT-6 AstraProvisional | 6.67 | 10.00 | 2 |
| GPT-6 SolProvisional | 10.00 | 10.00 | 2 |
| Google Gemini 4 ArgonProvisional | 8.34 | 10.00 | 2 |
| Meta Muse Spark 1.3Provisional | 10.00 | 10.00 | 2 |
| Meta Muse WonderProvisional | 8.34 | 10.00 | 2 |
Insight codification · 10 complete comparisons
| Model | Simple prompt | Scaffolded prompt | Paired observations |
|---|---|---|---|
| Claude Fable 5Provisional | 5.90 | 7.44 | 6 |
| Claude Fable 5.1Provisional | 4.79 | 5.63 | 1 |
| Claude Opus 5.5Provisional | 5.87 | 7.56 | 2 |
| Gemini 3.7 FlashProvisional | 6.90 | 7.57 | 6 |
| Gemini 3.8 FlashProvisional | 4.79 | 5.25 | 1 |
| GPT-6 AstraProvisional | 3.23 | 6.50 | 1 |
| GPT-6 SolProvisional | 6.65 | 5.98 | 2 |
| Google Gemini 4 ArgonProvisional | 4.52 | 7.44 | 2 |
| Meta Muse Spark 1.3Provisional | 7.13 | 7.44 | 2 |
| Meta Muse WonderProvisional | 5.27 | 7.71 | 2 |
How to read this
A quick guide to the numbers.
Time saved. Allow for checking and fixing the AI's work. A full redo can wipe out the saving. Without repair evidence, I count no saving. These timings are estimates.
Quality out of 10. Human-rated tasks compare outputs with skilled human work; automated tasks measure correctness against their own answer keys. Compare models within a task. Where both prompt setups are comparable, the stronger whole-test score is shown.
Still being tested. “Provisional” results are included but need more checking. “Published” or “final” means those checks are complete.
Overall scores. Each scored task counts equally in the model average. A dash in a score column means it hasn't been scored. Check coverage: models haven't all done the same tasks.
Updated as I score. New results can change the rankings. The date shows the latest scored result.
What a scaffold adds. I test the same task with a plain prompt and extra guidance. The headline averages the tasks where that guidance helped, with each improved task counting equally. Losses and unchanged results are excluded. Task details show both scores.