All evidence cases
DeepSWE v1.1repositoryPython

Add GraphQL incremental delivery with @defer and @stream

Add incremental GraphQL response handling with @defer and @stream across HTTP multipart, WebSocket transport, and the DSL.

gql-incremental-graphql-delivery

Normalized profile

What this task asks for

Add incremental GraphQL response handling with @defer and @stream across HTTP multipart, WebSocket transport, and the DSL. Add @defer and @stream directive support so servers can send critical data first while deferring or streaming non-essential fields incrementally. Support nested paths navigating through lists by index, null values, field overwrites, and concurrent deferred/streamed fields. Empty incremental arrays and hasNext-only payloads (without data or incremental fields) must still yield a result. Errors must not halt subsequent items. Both HTTP multipart (boundary=graphql, deferSpec=20220824) and WebSocket transports must support incremental delivery. The WebSocket transport must forward incremental payloads through the existing protocol.

  • feature implementation
  • configuration tuning
  • query layer
  • network service
  • task scheduler
  • gql
  • GraphQL
  • HTTP
  • WebSocket
  • repository patch
  • concurrency, cancellation, or distributed-state coordination
  • specialized algorithmic or domain reasoning
  • changes span multiple components or interfaces

Published evidence

Model × harness outcomes

Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.

Models
13
Configurations
41
Trials
164

Showing 41 of 41 configurations

ModelHarnessEffortResultPasses
claude-fable-5vertex_aimini-swe-agentpublished configurationlowNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmediumNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationhighNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationxhighNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmaxNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationlowNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmediumNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationhighMixed1/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationxhighNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmaxNo passes0/4
claude-sonnet-4-6anthropicmini-swe-agentpublished configurationhighNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationlowNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmediumNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationhighNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationxhighNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmaxMixed2/4
gemini-3-1-pro-previewvertex_aimini-swe-agentpublished configurationhighNo passes0/4
gemini-3-5-flashvertex_aimini-swe-agentpublished configurationmediumNo passes0/4
glm-5-2zaimini-swe-agentpublished configurationhighMixed1/4
glm-5-2zaimini-swe-agentpublished configurationmaxMixed2/4
gpt-5-4openaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmaxNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmaxNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmaxNo passes0/4
kimi-k2-7-codeopenroutermini-swe-agentpublished configurationdefaultNo passes0/4