blind, seeded, human-rated
80 generated garments, 4 conditions, rated by people who cannot see which condition produced what
80 images, four conditions, a frozen rubric
Everyone building on image models shows you the good outputs. So I ran the opposite test.
Eighty garment images across four generation conditions, presented to human raters in a
seeded shuffle, with condition and filenames masked and the rubric frozen before anyone
rated anything. The rubric gates on one question first: is this image clear enough to
judge at all. If the answer is no, the rest is skipped rather than guessed.
The interesting part is not the score. It is that an automatic OCR score and a human
judgement disagree often enough to matter, and the only way to find out which one is right
is to ask a person who cannot see the pipeline.
The instrument →
Every call behind it →