Identity comes before a clean filename
A folder can contain the same asset under different names, or different assets with almost identical names. Renaming them consistently is useful only after those relationships are resolved.
The benchmark supplies content hashes, existing registry entries, parent relationships and a complete policy. Its cases include duplicates, version regressions, placement derivatives and ambiguous parentage. The model must return a proposed operation for each source row, rather than improvise a naming scheme from filenames alone.
Judge the whole change set
A correct name does not compensate for an incorrect parent asset or a collision with another file. Benmark therefore checks exact components and critical conditions alongside the composite score, including idempotence and whole-set correctness.
This changes the workflow decision. Use a high-scoring model to prepare the mapping, then validate the mapping against the registry and naming rules. The approval unit should be the complete set of changes, not the average quality of its rows.
Make review straightforward
Present the current identity, proposed identity, reason for the operation and any unresolved ambiguity together. A reviewer should be able to see why an item is a new version, a derivative or a duplicate.
The benchmark is a dry run and does not rename files. For a practical implementation, retain that separation: generate and validate the proposal first, then use a controlled application step with a reversible mapping. This keeps the model useful without making its interpretation the sole record of what changed.
How Benmark tests this
Models reconcile asset metadata, content hashes, a registry and naming policy into a proposed change set. Two synthetic packs contain 48 and 24 operations. The scorer checks exact names, versions, lineage, duplicates and collision handling.
Test inputs, instructions and scoring
What the model receives
The model receives stable asset IDs, existing filenames, content hashes, version relationships, requested operations and the full naming policy. Cases include duplicates under different names, conflicting versions, case/Unicode collisions and ambiguous parentage. The 72 operation rows belong to two packs; repeat outputs are not additional packs.
What changes between prompts
Both conditions receive the complete policy, facts and output schema. The scaffold adds an ordered process for identity reconciliation, naming, lineage, collision checks and escalation. The output is a dry-run proposal; the evaluation does not rename files.
How the result is scored
A deterministic exact-key scorer assesses filenames, actions, versions, parentage, duplicates, exceptions and idempotence. The mean correctness score is accompanied by strict whole-change-set and critical-safety checks. An attractive average does not mean every proposed operation is safe to apply; whole-set acceptance is the operational decision.