Motion multiplies the decisions
A static image has to resolve one composition. A video has to preserve the product and message while the scene, camera or person changes. A strong opening frame cannot rescue a sequence that loses the product or fails to deliver the intended demonstration.
The benchmark therefore scores the complete unedited output. It covers UGC, founder, lifestyle, explainer and product-demonstration briefs, giving a more useful comparison than selecting each model's most impressive frame.
Use a genuinely shared input contract
The tested video setup is text only. Reference images were excluded because providers use them differently: a first frame and a general visual reference impose different tasks. All models receive a short common brief for an eight-second vertical output.
Audio is excluded from scoring. The result should guide model choice for the visual sequence under that contract. If your process depends on a fixed first frame, dialogue or several stitched shots, evaluate that production setup on its own terms.
Build the production process around the failure
Use an initial output to assess which part of the brief the model can execute: the product demonstration, the setting, the motion or the transition. Then decide whether the next step is a clearer brief, a narrower shot or conventional editing.
The benchmark has not measured a multi-shot edited workflow. It gives you evidence for choosing the generation model and a concrete set of review dimensions, which is a better starting point than assuming a strong demo clip will reproduce reliably on your brief.
How Benmark tests this
Video models create complete eight-second vertical ads from five briefs. Both prompt conditions use text only, and Ben scores the unedited video; audio is excluded.
Test inputs, instructions and scoring
What the model receives
The briefs span UGC, founder, lifestyle, explainer and product-demonstration formats. The shared contract requests an eight-second 9:16 video at 720×1280. No reference images are supplied because providers interpret them differently. Five briefs in two conditions produce ten outputs per run.
What changes between prompts
The simple and scaffolded prompts share the same task. The scaffold adds concise ad-anatomy guidance within the common prompt limit. Video duration, geometry and decodability are checked before the result is assessed; scoring playback is muted.
How the result is scored
Raw outputs are human-scored for product fidelity, copy accuracy, claims safety, brief compliance and craft quality. Each dimension must reach 8 for an output to clear the usability threshold. The displayed mean is a comparison measure; it can conceal a weak individual dimension. No editing pass is added before scoring.