All evidence cases
DeepSWE v1.1repositoryTypeScript

Add pair-level relation tracking modifiers

Tracking modifiers should distinguish changes to specific relation pairs, not just trait-level additions and removals.

koota-pair-relation-tracking

Normalized profile

What this task asks for

Tracking modifiers should distinguish changes to specific relation pairs, not just trait-level additions and removals. Tracking modifiers detect trait-level additions and removals but cannot distinguish which specific relation pair changed, blocking per-target reactivity. Make tracking modifier factories accept new `RelationPair`. The target `'*'` acts as a wildcard. Pair modifiers compose with `Or`. Pair modifiers combined with regular trait parameters in the same query must satisfy all constraints together. The `entity.changed` method accepts a `RelationPair` for manual pair-level change signaling.

  • feature implementation
  • performance optimization
  • data transformation
  • cache
  • query layer
  • library API
  • Koota
  • repository patch
  • composite state and lifecycle-transition semantics
  • changes span multiple components or interfaces

Published evidence

Model × harness outcomes

Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.

Models
13
Configurations
41
Trials
164

Showing 41 of 41 configurations

ModelHarnessEffortResultPasses
claude-fable-5vertex_aimini-swe-agentpublished configurationlowMixed3/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmediumMixed2/4
claude-fable-5vertex_aimini-swe-agentpublished configurationhighMixed3/4
claude-fable-5vertex_aimini-swe-agentpublished configurationxhighAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmaxAll passed4/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationlowNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmediumNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationhighNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationxhighNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmaxMixed1/4
claude-sonnet-4-6anthropicmini-swe-agentpublished configurationhighNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationlowMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmediumMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationhighMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationxhighNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmaxNo passes0/4
gemini-3-1-pro-previewvertex_aimini-swe-agentpublished configurationhighNo passes0/4
gemini-3-5-flashvertex_aimini-swe-agentpublished configurationmediumNo passes0/4
glm-5-2zaimini-swe-agentpublished configurationhighNo passes0/4
glm-5-2zaimini-swe-agentpublished configurationmaxNo passes0/4
gpt-5-4openaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationmediumMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationhighMixed2/4
gpt-5-5openaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmaxMixed3/4
gpt-5-6-solopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmaxNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmaxNo passes0/4
kimi-k2-7-codeopenroutermini-swe-agentpublished configurationdefaultNo passes0/4