All evidence cases
DeepSWE v1.1repositoryTypeScript

Add config file parsing to Cliffy commands

Add command-level config file loading, parsing, merging, and precedence handling.

cliffy-config-file-parsing

Normalized profile

What this task asks for

Add command-level config file loading, parsing, merging, and precedence handling. When searchPaths is not provided, the current directory is used. The RC format uses key=value pairs per line where lines starting with # are comments, empty lines are ignored, and values in double quotes preserve spaces. RC values are coerced to match option types where true/false become booleans and numeric strings become numbers. The getConfigValues method returns an empty object when no config is found. When mergeConfigs is false (the default), only the first matching config file is used. When mergeConfigs is true, configs from all search paths are merged with earlier paths taking precedence.

  • feature implementation
  • performance optimization
  • configuration tuning
  • data transformation
  • CLI
  • parser
  • cache
  • serialization
  • configuration
  • Cliffy
  • JSON
  • repository patch
  • strict behavioral compatibility or exact-output validation
  • layered configuration precedence and inheritance semantics
  • changes span multiple components or interfaces

Published evidence

Model × harness outcomes

Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.

Models
13
Configurations
41
Trials
164

Showing 41 of 41 configurations

ModelHarnessEffortResultPasses
claude-fable-5vertex_aimini-swe-agentpublished configurationlowMixed2/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmediumAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationhighAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationxhighAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmaxAll passed4/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationlowMixed2/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmediumMixed3/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationhighNo passes0/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationxhighMixed1/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmaxMixed3/4
claude-sonnet-4-6anthropicmini-swe-agentpublished configurationhighMixed3/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationlowMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmediumMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationhighMixed1/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationxhighMixed2/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmaxMixed1/4
gemini-3-1-pro-previewvertex_aimini-swe-agentpublished configurationhighNo passes0/4
gemini-3-5-flashvertex_aimini-swe-agentpublished configurationmediumNo passes0/4
glm-5-2zaimini-swe-agentpublished configurationhighMixed1/4
glm-5-2zaimini-swe-agentpublished configurationmaxMixed2/4
gpt-5-4openaimini-swe-agentpublished configurationxhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationlowMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationhighNo passes0/4
gpt-5-5openaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationhighMixed2/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationxhighMixed1/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmaxMixed2/4
gpt-5-6-solopenaimini-swe-agentpublished configurationlowAll passed4/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmediumMixed2/4
gpt-5-6-solopenaimini-swe-agentpublished configurationhighMixed2/4
gpt-5-6-solopenaimini-swe-agentpublished configurationxhighAll passed4/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmaxMixed3/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmediumMixed3/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationhighMixed3/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationxhighAll passed4/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmaxAll passed4/4
kimi-k2-7-codeopenroutermini-swe-agentpublished configurationdefaultNo passes0/4