Add policy-based alerting for failures, latency, and SSL expiry
Add policy-driven alerting with consecutive failure, latency, recovery, and SSL expiry notifications.
updo-policy-alerting
Normalized profile
What this task asks for
Add policy-driven alerting with consecutive failure, latency, recovery, and SSL expiry notifications. While a target remains degraded, every later slow check should produce `target_degraded`. Suppression affects delivery, not evaluation: `Decision` must still report the state change and set `Suppressed=true`. Each evaluation should return a current snapshot: `State`, `PreviousState`, `ConsecutiveFailures`, `ConsecutiveRecoveries`, `LatencyBreaches`, and `SSLDaysRemaining` should match tracker state even when `Event == EventNone` or `Suppressed == true`. `alerts.NewTracker(Policy)` must return a tracker with `Evaluate(Check, time.Time) Decision`. `HandleWebhookDecisionWithHeaders` must preserve custom headers. Decision webhook helpers must not send when `decision.Event == EventNone` or `decision.Suppressed == true`.
Intent
- feature implementation
- configuration tuning
- data transformation
- bug repair
Work surfaces
- serialization
- configuration
Technology
- Updo
- JSON
Expected artifacts
- repository patch
What makes it difficult
- reverse engineering from incomplete or indirect evidence
- strict behavioral compatibility or exact-output validation
- quantitative performance, accuracy, or resource constraints
Published evidence
Model × harness outcomes
Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.
- Models
- 13
- Configurations
- 41
- Trials
- 164
Showing 41 of 41 configurations
| Model | Harness | Effort | Result | Passes |
|---|---|---|---|---|
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | xhigh | No passes | 0/4 |
| claude-fable-5vertex_ai | mini-swe-agentpublished configuration | max | No passes | 0/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | xhigh | No passes | 0/4 |
| claude-opus-4-8anthropic | mini-swe-agentpublished configuration | max | No passes | 0/4 |
| claude-sonnet-4-6anthropic | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | xhigh | No passes | 0/4 |
| claude-sonnet-5anthropic | mini-swe-agentpublished configuration | max | No passes | 0/4 |
| gemini-3-1-pro-previewvertex_ai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| gemini-3-5-flashvertex_ai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| glm-5-2zai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| glm-5-2zai | mini-swe-agentpublished configuration | max | No passes | 0/4 |
| gpt-5-4openai | mini-swe-agentpublished configuration | xhigh | No passes | 0/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | high | Mixed | 2/4 |
| gpt-5-5openai | mini-swe-agentpublished configuration | xhigh | Mixed | 1/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | medium | Mixed | 1/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | xhigh | Mixed | 1/4 |
| gpt-5-6-lunaopenai | mini-swe-agentpublished configuration | max | Mixed | 1/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | low | Mixed | 1/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | medium | Mixed | 2/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | high | Mixed | 2/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | xhigh | Mixed | 3/4 |
| gpt-5-6-solopenai | mini-swe-agentpublished configuration | max | All passed | 4/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | low | No passes | 0/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | medium | No passes | 0/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | high | No passes | 0/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | xhigh | No passes | 0/4 |
| gpt-5-6-terraopenai | mini-swe-agentpublished configuration | max | Mixed | 1/4 |
| kimi-k2-7-codeopenrouter | mini-swe-agentpublished configuration | default | No passes | 0/4 |