Production brief generation: benchmark methods and results · 16 October 2026
Models turn a fixed concept and brand/production constraints into a structured brief. The public task score measures brief quality with a compliance gate; hook and narrative quality are assessed separately.
Reviewed 16 October 2026 · Tests and editorial direction: Ben Nancarrow · AI-assisted writing and checks.
The input supplies the creative starting point, brand context and production constraints. The output should give production a grounded, actionable handover. Recorded model rows can combine more than one completed cycle; the results table preserves their output counts.
What changes between prompts
The same fixed creative input and brand context are used with plain and scaffolded-context prompts. The structured condition adds guidance for developing the output. The publisher keeps hooks, narratives and compliant briefs as separate tasks, with their own scores.
How the result is scored
The published compliant-brief task uses the brief-quality component and applies compliance as a gate. Its scorer is AI-assisted within the reviewed scoring process. Human hook and narrative scores exist in the wider brief evaluation but are deliberately excluded from this task's headline. A 9.5 here therefore describes the handover, not a 9.5 advertising idea.
All 16 reviewed model results
Compliance-gated brief quality / 10
Model
Result / setup
Scored outputs
Configuration / recorded test date
Cost / scored output
Claude Fable 5published
9.50/10Simple prompt
2
thinking_max23 August 2026
Not recorded
Claude Fable 5.1published
9.50/10Simple prompt
2
thinking_max9 September 2026
Not recorded
Claude Opus 5published
9.50/10Scaffolded prompt
2
thinking_max23 August 2026
Not recorded
Claude Opus 5.5published
9.50/10Simple prompt
2
thinking_max23 September 2026
Not recorded
Gemini 3.8 Flashpublished
9.50/10Scaffolded prompt
2
reasoning_high9 September 2026
Not recorded
GPT-5.6 Solpublished
9.50/10Scaffolded prompt
2
xhigh23 August 2026
Not recorded
GPT-6 Astrapublished
9.50/10Scaffolded prompt
2
xhigh9 September 2026
Not recorded
GPT-6 Solpublished
9.50/10Simple prompt
2
xhigh23 September 2026
Not recorded
Meta Muse Spark 1.3published
9.50/10Scaffolded prompt
2
max9 September 2026
Not recorded
Meta Muse Wonderpublished
9.50/10Simple prompt
2
max14 October 2026
Not recorded
Claude Opus 4.8provisional
9.39/10Scaffolded prompt
12
thinking_high20 August 2026
Not recorded
GPT-5.5published
9.38/10Scaffolded prompt
20
Not recorded26 May 2026
Not recorded
Gemini 3.1 Pro Previewpublished
9.37/10Scaffolded prompt
10
Not recorded26 May 2026
Not recorded
Gemini 3.7 Flashpublished
9.17/10Scaffolded prompt
2
reasoning_high23 August 2026
Not recorded
Google Gemini 4 Argonpublished
9.17/10Simple prompt
2
high14 October 2026
Not recorded
Gemini 3.5 Flashpublished
8.81/10Simple prompt
14
Not recorded29 July 2026
Not recorded
Recorded output counts can include repeats and both prompt conditions. Configurations and cost bases are retained; joint scores remain joint.
This record preserves the evidence and interpretation reviewed together. The live explorer can advance independently. Public results include provisional evidence; the methods above define what the scores measure.