Terminal-Bench 2.1terminalSQL
Optimize an Open English WordNet SQL query
Rewrite the supplied SQLite query for maximum efficiency while preserving its exact result against the immutable OEWN database.
query-optimize
Normalized profile
What this task asks for
Rewrite the supplied SQLite query for maximum efficiency while preserving its exact result against the immutable OEWN database. Preserve the exact output of /app/my-sql-query.sql. Leave /app/oewn.sqlite unchanged. Use SQLite syntax. Write one comment-free query terminated by a semicolon.
Intent
- performance optimization
- data transformation
- test and validation
Work surfaces
- database query
- query plan
Technology
- SQLite
- Open English WordNet
- SQL
Expected artifacts
- /app/sol.sql
What makes it difficult
- strict behavioral compatibility or exact-output validation
- specialized algorithmic or domain reasoning
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 20
- Trials
- 100
Showing 20 of 20 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5anthropic · 2026-06-07 | claude-code2.1.167 | xhigh | Mixed2 errors | 1/5 |
| claude-fable-5anthropic · 2026-06-05 | terminus-22.0.0 | high | Mixed | 4/5 |
| claude-opus-4-7anthropic · 2026-05-01 | claude-code2.1.123 | max | Mixed3 errors | 2/5 |
| claude-opus-4-7anthropic · 2026-05-01 | terminus-22.0.0 | max | Mixed | 4/5 |
| claude-opus-4-8anthropic · 2026-07-09 | claude-code2.1.205 | high | No passes4 errors | 0/5 |
| claude-sonnet-5anthropic · 2026-07-09 | claude-code2.1.205 | high | Mixed | 3/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | gemini-cli0.40.0 | high | Mixed1 errors | 3/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | terminus-22.0.0 | high | Mixed | 3/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | gemini-cli0.40.0 | high | Mixed4 errors | 1/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | terminus-22.0.0 | high | Mixed1 errors | 3/5 |
| glm-5.1zai · 2026-05-01 | claude-code2.1.123 | max | Mixed | 1/5 |
| gpt-5.5openai · 2026-05-01 | codex0.125.0 | xhigh | Mixed1 errors | 4/5 |
| gpt-5.5openai · 2026-05-01 | terminus-22.0.0 | xhigh | Mixed2 errors | 3/5 |
| gpt-5.6-lunaopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-lunaopenai · 2026-07-11 | codex0.144.1 | max | Mixed1 errors | 3/5 |
| gpt-5.6-solopenai · 2026-07-10 | codex0.144.0 | max | Mixed1 errors | 4/5 |
| gpt-5.6-terraopenai · 2026-07-10 | codex0.144.0 | max | Mixed3 errors | 1/5 |
| gpt-5.6-terraopenai · 2026-07-11 | codex0.144.1 | max | Mixed1 errors | 3/5 |
| grok-4.5cursor · 2026-07-09 | cursor-cli2026.07.08-0c04a8a | default | Mixed | 3/5 |
| muse-spark-1.1openai · 2026-07-09 | mini-swe-agent2.4.5 | xhigh | All passed | 5/5 |