First drafts are rarely great — for humans or for models. The evaluator–optimizer pattern pairs two roles: a generator produces an attempt, an evaluator critiques it against explicit criteria, and the generator retries with that feedback. Loop until it passes (or you hit a retry cap).

The analogy

The writer and the editor. The novelist writes a chapter; the editor sends it back covered in red ink: “the dialogue drags, the timeline is off.” Second version, better. The trick is that criticizing is easier than creating — a mediocre writer can still be an excellent judge of what’s wrong.

The principle

flowchart LR
    IN([input]) --> G["LLM: generate"] --> E["LLM: evaluate"]
    E -->|feedback| G
    E -->|pass| OUT([output])
  • The evaluator gets a rubric: precise, checkable criteria (“all claims sourced?”, “compiles?”, “under 200 words?”).
  • Feedback is actionable, not a score: “the second paragraph contradicts the first” beats “6/10”.
  • A cap (2–3 iterations) prevents infinite loops; whatever still fails then escalates to a human.

A concrete example

Generating a tricky SQL migration:

1. generator → writes the migration script
2. evaluator → "column rename loses data on rollback;
                index creation will lock the table"
3. generator → rewrites with a safe two-step rename
4. evaluator → all checks pass → done

The evaluator can even run things (linter, tests, dry-run) and feed real errors back — the strongest form of critique.

When to use it

  • Quality is verifiable: you can state what “good” means as concrete criteria.
  • First attempts are usually almost right and visibly improve with targeted feedback (writing, code, translations).
  • You’d rather spend 2–3× the tokens than ship a mediocre first draft.

When to avoid it

  • No one can articulate the criteria — the evaluator will critique vibes and the loop will wander.
  • The task is easy: looping over an already-fine answer burns money to change adjectives.

The classic trap

The complacent evaluator. If the same model critiques its own fresh output with no rubric, it tends to say “looks good!”. Give the evaluator a different role, explicit criteria — and ideally real tools to verify, not just opinions.