Fix PromQL label sorting across typed and untyped values
PromQL label sorting must order mixed typed and untyped label values with stable typed comparison rules.
prometheus-typed-label-sorting
Normalized profile
What this task asks for
PromQL label sorting must order mixed typed and untyped label values with stable typed comparison rules. Label sorting must use multi-domain typed comparison. Values with leading whitespace are never parsed as any typed form, and must sort before all other values. Numeric parsing must accept scientific exponents and optional leading plus signs. Duration and byte parsing must also support signed coefficients and scientific-notation magnitudes. All magnitude comparisons must preserve order for arbitrarily large values without loss of precision. IP and CIDR comparisons must place IPv4 values before IPv6 values.
Intent
- feature implementation
- bug repair
- configuration tuning
Work surfaces
- network service
Technology
- Prometheus
Expected artifacts
- repository patch
What makes it difficult
- strict behavioral compatibility or exact-output validation
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 41
- Trials
- 164
Showing 41 of 41 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | low | Mixed | 2/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | medium | Mixed | 3/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | high | Mixed | 3/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | xhigh | Mixed | 1/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | max | Mixed | 2/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | low | Mixed | 1/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | medium | Mixed | 1/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | high | Mixed | 1/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | xhigh | Mixed | 1/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | max | Mixed | 2/4 |
| claude-sonnet-4-6anthropic | mini-swe-agentpublished configuration | high | Mixed | 1/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | low | Mixed | 1/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | high | Mixed | 1/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | xhigh | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | max | No passes | 0/4 |
| gemini-3-1-pro-previewvertex_ai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| gemini-3-5-flashvertex_ai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| glm-5-2zai | mini-swe-agentpublished configuration | high | Mixed | 1/4 |
| glm-5-2zai | mini-swe-agentpublished configuration | max | Mixed | 1/4 |
| gpt-5-4openai | mini-swe-agentpublished configuration | xhigh | Mixed | 3/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | low | Mixed | 1/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | medium | Mixed | 3/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | high | Mixed | 3/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | xhigh | All passed | 4/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | high | Mixed | 1/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | xhigh | Mixed | 3/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | max | Mixed | 3/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | low | All passed | 4/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | medium | All passed | 4/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | high | All passed | 4/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | xhigh | Mixed | 3/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | max | All passed | 4/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | low | Mixed | 3/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | medium | Mixed | 1/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | high | Mixed | 3/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | xhigh | Mixed | 3/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | max | All passed | 4/4 |
| kimi-k2-7-codeopenrouter | mini-swe-agentpublished configuration | default | No passes | 0/4 |