All evidence cases
DeepSWE v1.1repositoryTypeScript

Add async autocomplete options and fetch lifecycle handling

Add async option fetching with caching, retries, debouncing, and loading state to AutocompletePrompt.

clack-async-autocomplete-options

Normalized profile

What this task asks for

Add async option fetching with caching, retries, debouncing, and loading state to AutocompletePrompt. Options must support existing forms (static array and synchronous function) without changing current behavior, plus async results. A loading property must be true while a fetch is in flight. Re-renders must only occur when the prompt is active (not during construction). Only the latest fetch result may be applied; stale results must not update state. A non-SWR cache hit or entering searchTooShort must invalidate any in-flight fetch (abort its signal and discard its pending result). Starting a new fetch must abort the previous signal. Errors with name 'AbortError' must be silently ignored (set loading to false, return without setting loadError). Non-abort failures must set loadError to a string. Fetches must be debounced by configurable debounceMs, defaulting to a sensible value (100-300ms) when omitted. Optional cacheResults with maxCacheSize and clearCache() must avoid redundant fetches.

  • feature implementation
  • performance optimization
  • configuration tuning
  • test and validation
  • cache
  • configuration
  • renderer
  • library API
  • Clack
  • repository patch
  • concurrency, cancellation, or distributed-state coordination
  • specialized algorithmic or domain reasoning
  • changes span multiple components or interfaces

Published evidence

Model × harness outcomes

Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.

Models
13
Configurations
41
Trials
164

Showing 41 of 41 configurations

ModelHarnessEffortResultPasses
claude-fable-5vertex_aimini-swe-agentpublished configurationlowMixed3/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmediumMixed3/4
claude-fable-5vertex_aimini-swe-agentpublished configurationhighMixed2/4
claude-fable-5vertex_aimini-swe-agentpublished configurationxhighMixed3/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmaxMixed2/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationlowNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmediumMixed1/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationhighMixed2/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationxhighMixed1/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmaxMixed2/4
claude-sonnet-4-6anthropicmini-swe-agentpublished configurationhighMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationlowNo passes0/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmediumMixed2/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationhighMixed3/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationxhighMixed2/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmaxMixed2/4
gemini-3-1-pro-previewvertex_aimini-swe-agentpublished configurationhighNo passes0/4
gemini-3-5-flashvertex_aimini-swe-agentpublished configurationmediumMixed3/4
glm-5-2zaimini-swe-agentpublished configurationhighMixed1/4
glm-5-2zaimini-swe-agentpublished configurationmaxMixed1/4
gpt-5-4openaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationlowMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationmediumMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationhighMixed1/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmaxNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationlowMixed1/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmediumMixed1/4
gpt-5-6-solopenaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmaxNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmediumMixed2/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationhighMixed3/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmaxNo passes0/4
kimi-k2-7-codeopenroutermini-swe-agentpublished configurationdefaultNo passes0/4