Terminal-Bench 2.1terminalPythonCCython
Build pyknotid Cython extensions for NumPy 2.3
Patch and install pyknotid 0.5.3 from source so its three Cython extensions work with NumPy 2.3.0 in the global Python environment.
build-cython-ext
Normalized profile
What this task asks for
Patch and install pyknotid 0.5.3 from source so its three Cython extensions work with NumPy 2.3.0 in the global Python environment. Clone branch 0.5.3 into /app/pyknotid. Compile chelpers, ccomplexity, and cinvariants. Preserve the package structure. Pass core tests except the two explicitly excluded files.
Intent
- bug repair
- systems setup
- test and validation
Work surfaces
- package build
- native extensions
- global Python environment
Technology
- pyknotid
- Cython
- NumPy 2.3.0
- Python packaging
Expected artifacts
- /app/pyknotid
- installed pyknotid package
What makes it difficult
- strict behavioral compatibility or exact-output validation
- native compilation and dependency integration
- environment provisioning and end-to-end integration
- cross-language or cross-format integration
- changes span multiple components or interfaces
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 20
- Trials
- 100
Showing 20 of 20 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5anthropic · 2026-06-07 | claude-code2.1.167 | xhigh | Mixed1 errors | 4/5 |
| claude-fable-5anthropic · 2026-06-05 | terminus-22.0.0 | high | Mixed2 errors | 3/5 |
| claude-opus-4-7anthropic · 2026-05-01 | claude-code2.1.123 | max | All passed | 5/5 |
| claude-opus-4-7anthropic · 2026-05-01 | terminus-22.0.0 | max | Mixed3 errors | 2/5 |
| claude-opus-4-8anthropic · 2026-07-09 | claude-code2.1.205 | high | All passed | 5/5 |
| claude-sonnet-5anthropic · 2026-07-09 | claude-code2.1.205 | high | Mixed | 4/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | gemini-cli0.40.0 | high | Mixed | 3/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | terminus-22.0.0 | high | Mixed | 3/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | gemini-cli0.40.0 | high | Mixed | 3/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | terminus-22.0.0 | high | Mixed | 1/5 |
| glm-5.1zai · 2026-05-01 | claude-code2.1.123 | max | All passed | 5/5 |
| gpt-5.5openai · 2026-05-01 | codex0.125.0 | xhigh | All passed | 5/5 |
| gpt-5.5openai · 2026-05-01 | terminus-22.0.0 | xhigh | Mixed | 4/5 |
| gpt-5.6-lunaopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-lunaopenai · 2026-07-11 | codex0.144.1 | max | All passed | 5/5 |
| gpt-5.6-solopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-terraopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-terraopenai · 2026-07-11 | codex0.144.1 | max | All passed | 5/5 |
| grok-4.5cursor · 2026-07-09 | cursor-cli2026.07.08-0c04a8a | default | All passed | 5/5 |
| muse-spark-1.1openai · 2026-07-09 | mini-swe-agent2.4.5 | xhigh | All passed | 5/5 |