TEST · 4 SEP 2026

I use a vision model as a measuring instrument: it looks at each generated image and fills in a small form — camera height, camera pitch, and so on. The research question behind it was “which phrasings produce a usable low-angle shot?” When I scored the instrument against 30 images I had labelled by hand, one line in its prompt looked suspicious.

The first sentence the model received was: Research question this measurement serves: …, followed by the question. The function's own docstring said nothing but the image should go in. I had written the principle and broken it two lines below.

In effect I had told the judge what I hoped to find and then asked, “is this it?” The scores showed it. Low-angle recall was 100% on every version of the reader, which sounds wonderful, and precision was between 9.5% and 12.5%. The model wasn't finding low angles. It was calling nearly everything a low angle, so it couldn't miss one. Up to half of the high-angle photos got labelled low.

Agreement with my labels was 44.4% without the question and 11.1% with it. Twenty-five of the 30 images changed their answer depending on that one sentence. What matters more is the direction: every change moved toward “low angle,” and none went the other way. With the question, the model produced 13 low angles; without it, zero. Four of the five high-angle calls were erased into “unclear.”

The version without the question isn't good either. 44.4% is far below the 80% threshold I'd set, and it missed both of the real low-angle images. But the two failures differ in kind. The leaked version is biased: it pushes results in a direction, which flips conclusions. The clean version is merely dull, which weakens them. If there is an order in which to fix things, biased comes first.

The leaked version is biased: it pushes results in a direction, which flips conclusions. The clean version is merely dull, which weakens them.

This is an excerpt. The full write-up was first published on Haenginro Lab.