Terminal-Bench 2.1terminalINIemail
Configure a confirmed-opt-in Mailman list
Configure Postfix and Mailman 3 for a local reading-group list with confirmed join and leave messages and subscriber announcements delivered to Unix mailboxes.
mailman
Normalized profile
What this task asks for
Configure Postfix and Mailman 3 for a local reading-group list with confirmed join and leave messages and subscriber announcements delivered to Unix mailboxes. Support join and leave addresses with user confirmation. Deliver announcements to all subscribers without owner approval. Map <user>@local.edu to /var/mail/<username>. Set SubscriptionPolicy.open.
Intent
- systems setup
- configuration tuning
Work surfaces
- mailing list
- mail transport
- local mailboxes
Technology
- Mailman 3
- Postfix
- mbox
Expected artifacts
- /etc/mailman3/mailman.cfg
- reading-group@local.edu mailing list
What makes it difficult
- cross-language or cross-format integration
- changes span multiple components or interfaces
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 20
- Trials
- 100
Showing 20 of 20 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5anthropic · 2026-06-07 | claude-code2.1.167 | xhigh | All passed | 5/5 |
| claude-fable-5anthropic · 2026-06-05 | terminus-22.0.0 | high | All passed | 5/5 |
| claude-opus-4-7anthropic · 2026-05-01 | claude-code2.1.123 | max | Mixed1 errors | 3/5 |
| claude-opus-4-7anthropic · 2026-05-01 | terminus-22.0.0 | max | All passed | 5/5 |
| claude-opus-4-8anthropic · 2026-07-09 | claude-code2.1.205 | high | All passed | 5/5 |
| claude-sonnet-5anthropic · 2026-07-09 | claude-code2.1.205 | high | All passed | 5/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | gemini-cli0.40.0 | high | Mixed | 2/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | terminus-22.0.0 | high | Mixed | 1/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | gemini-cli0.40.0 | high | Mixed2 errors | 3/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | terminus-22.0.0 | high | Mixed | 2/5 |
| glm-5.1zai · 2026-05-01 | claude-code2.1.123 | max | All passed | 5/5 |
| gpt-5.5openai · 2026-05-01 | codex0.125.0 | xhigh | All passed | 5/5 |
| gpt-5.5openai · 2026-05-01 | terminus-22.0.0 | xhigh | Mixed1 errors | 2/5 |
| gpt-5.6-lunaopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-lunaopenai · 2026-07-11 | codex0.144.1 | max | All passed | 5/5 |
| gpt-5.6-solopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-terraopenai · 2026-07-10 | codex0.144.0 | max | All passed | 5/5 |
| gpt-5.6-terraopenai · 2026-07-11 | codex0.144.1 | max | All passed | 5/5 |
| grok-4.5cursor · 2026-07-09 | cursor-cli2026.07.08-0c04a8a | default | All passed | 5/5 |
| muse-spark-1.1openai · 2026-07-09 | mini-swe-agent2.4.5 | xhigh | No passes5 errors | 0/5 |