Implement row and column tensor-parallel linear layers
Implement distributed PyTorch linear layers with correct master-weight sharding, collective-style outputs, zero bias initialization, and gradient behavior.
torch-tensor-parallelism
Normalized profile
What this task asks for
Implement distributed PyTorch linear layers with correct master-weight sharding, collective-style outputs, zero bias initialization, and gradient behavior. Export ColumnParallelLinear and RowParallelLinear with the specified constructors. Shard column weights and bias by output dimension then concatenate outputs. Shard row weights by input dimension then sum partial outputs while keeping full bias. Split the initialized master weight by rank. Support world sizes one, two, and four with correct gradients.
Intent
- feature implementation
- performance optimization
- test and validation
Work surfaces
- distributed linear layers
- weight shards
- gradient computation
Technology
- PyTorch
- torch.distributed
- tensor parallelism
Expected artifacts
- /app/parallel_linear.py
What makes it difficult
- concurrency, cancellation, or distributed-state coordination
- changes span multiple components or interfaces
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 20
- Trials
- 100
Showing 20 of 20 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5anthropic · 2026-06-07 | claude-code2.1.167 | xhigh | All passed | 5/5 |
| claude-fable-5anthropic · 2026-06-05 | terminus-22.0.0 | high | All passed | 5/5 |
| claude-opus-4-7anthropic · 2026-05-01 | claude-code2.1.123 | max | Mixed | 4/5 |
| claude-opus-4-7anthropic · 2026-05-01 | terminus-22.0.0 | max | Mixed1 errors | 3/5 |
| claude-opus-4-8anthropic · 2026-07-09 | claude-code2.1.205 | high | All passed | 5/5 |
| claude-sonnet-5anthropic · 2026-07-09 | claude-code2.1.205 | high | Mixed | 4/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | gemini-cli0.40.0 | high | All passed | 5/5 |
| gemini-3-pro-previewgemini · 2026-05-01 | terminus-22.0.0 | high | All passed | 5/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | gemini-cli0.40.0 | high | All passed | 5/5 |
| gemini-3.1-pro-previewgemini · 2026-05-05 | terminus-22.0.0 | high | Mixed1 errors | 4/5 |
| glm-5.1zai · 2026-05-01 | claude-code2.1.123 | max | All passed | 5/5 |
| gpt-5.5openai · 2026-05-01 | codex0.125.0 | xhigh | Mixed | 4/5 |
| gpt-5.5openai · 2026-05-01 | terminus-22.0.0 | xhigh | All passed | 5/5 |
| gpt-5.6-lunaopenai · 2026-07-10 | codex0.144.0 | max | Mixed2 disqualified | 1/5 |
| gpt-5.6-lunaopenai · 2026-07-11 | codex0.144.1 | max | No passes | 0/5 |
| gpt-5.6-solopenai · 2026-07-10 | codex0.144.0 | max | Mixed1 disqualified | 4/5 |
| gpt-5.6-terraopenai · 2026-07-10 | codex0.144.0 | max | Mixed2 disqualified | 2/5 |
| gpt-5.6-terraopenai · 2026-07-11 | codex0.144.1 | max | Mixed | 4/5 |
| grok-4.5cursor · 2026-07-09 | cursor-cli2026.07.08-0c04a8a | default | Mixed | 4/5 |
| muse-spark-1.1openai · 2026-07-09 | mini-swe-agent2.4.5 | xhigh | Mixed | 1/5 |