Add grouped test phases with synchronized barriers
Add grouped execution phases with explicit synchronization barriers and per-group setup and teardown.
mobly-grouped-test-barriers
Normalized profile
What this task asks for
Add grouped execution phases with explicit synchronization barriers and per-group setup and teardown. Run global setup and teardown around no-entry, implicit-group, or explicit-group execution modes. Run explicit group participants concurrently between one group setup and one group teardown. Resolve participant group and ID from configuration entries and expose device context only in group phases and tests. Synchronize explicit participants by instance, group, hook or test, and step name with reusable barriers. Reject invalid synchronization contexts and timeouts with the documented exception types and messages. Continue other groups after group setup failures and always run the applicable teardown hooks.
Intent
- feature implementation
- bug repair
- test and validation
Work surfaces
- configuration
- task scheduler
- library API
- build system
Technology
- Mobly
Expected artifacts
- repository patch
What makes it difficult
- concurrency, cancellation, or distributed-state coordination
- quantitative performance, accuracy, or resource constraints
- consistent behavior across lifecycle stages or execution modes
- changes span multiple components or interfaces
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 41
- Trials
- 164
Showing 41 of 41 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | low | All passed | 4/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | medium | Mixed | 2/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | high | All passed | 4/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | xhigh | Mixed | 3/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | max | All passed | 4/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | low | Mixed | 1/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | medium | Mixed | 1/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | high | Mixed | 2/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | xhigh | Mixed | 2/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | max | Mixed | 2/4 |
| claude-sonnet-4-6anthropic | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | low | Mixed | 2/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | medium | All passed | 4/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | high | All passed | 4/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | max | Mixed | 3/4 |
| gemini-3-1-pro-previewvertex_ai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| gemini-3-5-flashvertex_ai | mini-swe-agentpublished configuration | medium | Mixed | 3/4 |
| glm-5-2zai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| glm-5-2zai | mini-swe-agentpublished configuration | max | No passes | 0/4 |
| gpt-5-4openai | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | low | Mixed | 1/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | medium | Mixed | 3/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | high | All passed | 4/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | high | Mixed | 2/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | max | All passed | 4/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | low | Mixed | 2/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | medium | Mixed | 1/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | high | All passed | 4/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | max | All passed | 4/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | medium | Mixed | 2/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | max | Mixed | 2/4 |
| kimi-k2-7-codeopenrouter | mini-swe-agentpublished configuration | default | No passes | 0/4 |