Every 3D model this company sells passes through a quality gate before it is listed: the mesh is rendered from several angles and a vision-language model inspects the renders, with the authority to block the listing. For weeks this judge had the opposite of the problem you would expect — it was too kind. Genuinely broken geometry sailed through with compliments. So we did the obvious thing. We made the prompt stricter.
Into the judging prompt went the vocabulary of everything we feared: "melted-looking geometry," "reads as AI blob," "mushy silhouette." Look hard for these, we told it. Be severe.
The judge became severe. It began failing every single asset — one hundred percent, for days. And when we read its verdicts, there was our own vocabulary, handed back to us nearly word for word. A clean, crisp low-poly model with sharp edges and a coherent silhouette: "melted-looking, reads as AI blob." The judge had not become a stricter observer. It had become a mirror. We asked it to find blobs, so everything, dutifully, became a blob.
A language model given evaluative vocabulary does not treat it as a detection target; it treats it as a completion suggestion. The words you put into a judge prompt are not instructions about the world — they are the statistical gravity of the answer. Ask "does this read as an AI blob?" and you have massively increased the probability that the answer contains "AI blob," whatever the pixels say. Meanwhile the humans spot-checking the rejected queue rated most of the failed assets as fine, which is how the incident surfaced at all: the pipeline had quietly stopped shipping while reporting, with great confidence, that it was protecting us.
The judge prompt today contains no negative vocabulary at all. It may not call anything a blob, because the word does not appear. Instead it must produce a specific claim: name a defect, in a named part of the model, visible in a named render. "The left rear leg intersects the body at the hip" is admissible. A vibe is not. If the judge cannot articulate a concrete, located defect, the asset passes. Rejections became rare, and — for the first time — checkable, because every rejection now points at something a human can go look at.
Never put the verdict you fear into the mouth of the judge. It is the LLM-age version of leading the witness, and the witness is infinitely suggestible. Positive, falsifiable rubrics only: name the part, name the defect, name the view. And keep a human spot-check on the reject pile, not just the accept pile — a gate that blocks everything feels safe from the inside, and it is the most expensive failure a shipping company can have.
The AI news that matters — in your inbox by 07:30 CET. Free, no spam.