All evidence cases
DeepSWE v1.1repositoryPython

Add a persistent analysis cache to Vulture

Add a persistent cache so Vulture can reuse unchanged analysis across runs.

vulture-persistent-analysis-cache

Normalized profile

What this task asks for

Add a persistent cache so Vulture can reuse unchanged analysis across runs. Vulture scans every file from scratch on every run, making it slow on large codebases where only a few files changed. `--cache` and `--cache-clear` flags are added to the CLI, with an optional `--cache-dir=PATH` (default `.vulture-cache/`). `--cache-clear` removes all contents of the cache directory before running. The vulture package version must be obtained via `importlib.metadata.version`, and `importlib` must be imported at module scope in `vulture.cache`. Concurrent vulture processes must not corrupt the cache. On every successful save, both a backup of the cache (`cache.json.bak`) and a metadata hash file (`cache.json.meta`) must be written, even on the very first save.

  • feature implementation
  • performance optimization
  • configuration tuning
  • test and validation
  • CLI
  • cache
  • task scheduler
  • runtime
  • Vulture
  • JSON
  • repository patch
  • reverse engineering from incomplete or indirect evidence
  • concurrency, cancellation, or distributed-state coordination
  • strict behavioral compatibility or exact-output validation
  • changes span multiple components or interfaces

Published evidence

Model × harness outcomes

Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.

Models
13
Configurations
41
Trials
164

Showing 41 of 41 configurations

ModelHarnessEffortResultPasses
claude-fable-5vertex_aimini-swe-agentpublished configurationlowMixed1/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmediumNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationhighNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationxhighNo passes0/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmaxNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationlowNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmediumMixed1/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationhighNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationxhighNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmaxMixed1/4
claude-sonnet-4-6anthropicmini-swe-agentpublished configurationhighMixed2/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationlowMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmediumMixed2/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationhighMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationxhighAll passed4/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmaxMixed2/4
gemini-3-1-pro-previewvertex_aimini-swe-agentpublished configurationhighMixed1/4
gemini-3-5-flashvertex_aimini-swe-agentpublished configurationmediumMixed1/4
glm-5-2zaimini-swe-agentpublished configurationhighMixed3/4
glm-5-2zaimini-swe-agentpublished configurationmaxAll passed4/4
gpt-5-4openaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationlowMixed3/4
gpt-5-5openaimini-swe-agentpublished configurationmediumMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmediumMixed1/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationhighMixed3/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationxhighMixed3/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmaxMixed3/4
gpt-5-6-solopenaimini-swe-agentpublished configurationlowMixed2/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-solopenaimini-swe-agentpublished configurationhighMixed2/4
gpt-5-6-solopenaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmaxMixed1/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationlowMixed3/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmediumMixed1/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationhighMixed2/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmaxNo passes0/4
kimi-k2-7-codeopenroutermini-swe-agentpublished configurationdefaultMixed1/4