01Interpret
Understand the task
LLM or local parser
intent · tools · surfaces · constraints
Example task
Add a Python CLI feature, update configuration, and add regression tests.
- Intent
- Feature implementation
- Signals
- Python · CLI · config · tests
How it works
From a task description to an evidence-backed route.
The model interprets, then judges blinded routes in the lowest eligible effort tier. Benchmarks and a fixed verifier keep the evidence constraints exact.
The workflow
One task in. One honest decision out.
Hover a stage—or tap “Example”
02Retrieve
Find analogues
256-d embeddings + cosine search
comparable benchmark cases
Example corpus match
Add incremental cache controls to Bandit
- Matched
- Feature intent · Python · config
- Source
- DeepSWE v1.1
03Measure
Read outcomes
Published trials only
passes / attempts by exact config
Example evidence group
claude-fable-5 · xhigh
- Compatible cases
- 8
- Observed
- 29 / 32 passed
04Select
Judge, then verify
GPT-5.6 Sol decision + fixed verifier
verified route · estimate · abstention
coverage ≥ 3 cases
90% Wilson lower bound ≥ target
citations = complete support set
below target → labeled estimate
Example judge proposal
Choose candidate-02
- Citations
- 8 / 8 cases
- Verifier
- Eligible · complete
Eligible
Recommend
A verified route that clears the target.
Below target
Estimate
Provisional model guidance, explicitly not certified.
Insufficient
Abstain
No defensible observed route or valid model estimate.
LLM judgment: profile, rerank, then judge eligible routes. When none qualify, GPT-5.6 Sol can provide a clearly labeled advisory estimate.
Deterministic authority: compute outcome and reliability gates, enforce the effort tier, and verify complete citations. Semantic transfer is LLM-assisted.
Worked example
Follow one task through.
DeepSWE v1.1 snapshot
Task · 70% target
Add a Python CLI feature, update configuration, and add regression tests.
Retrieved
8 cases
Observed
29 / 32
Lower bound
79%
Route
claude-fable-5
xhigh · mini-swe-agent
The LLM can
Interpret, rerank, certify—or estimate.
It judges eligible routes or estimates from observed candidates and must cite every supporting case.
The verifier ensures
Gates, statistics, and citations stay exact.
If the judge is unavailable or invalid, deterministic selection takes over.
Current scope: repository and terminal software work. Similar cases are evidence—not proof—and reasoning effort remains a proxy until cost and latency are normalized.
Try the method