All evidence cases
DeepSWE v1.1repositoryTypeScript

Format BigQuery pipe syntax queries correctly

Add parsing and formatting support for BigQuery pipe syntax queries without changing traditional SQL formatting.

sql-formatter-bigquery-pipe-formatting

Normalized profile

What this task asks for

Add parsing and formatting support for BigQuery pipe syntax queries without changing traditional SQL formatting. BigQuery pipe syntax chains transformations via `|>` instead of nested clauses. Pipe queries start with standalone `FROM` and each subsequent `|>` step occupies its own line at base indentation. Clauses that the existing formatter treats as indented clauses (`WHERE`, `SELECT`, `ORDER BY`, `AGGREGATE`, `EXTEND`, `SET`, `DROP`) place their body on a new indented line after the keyword. Clauses that the existing formatter treats as one-line clauses (`LIMIT`, `JOIN` and its variants, `AS`) keep their content on the same line as the keyword. Pipe-exclusive clauses absent from standard SQL include `AGGREGATE` with an optional nested `GROUP BY` sub-clause requiring its own indentation level, `EXTEND` for computed columns, `SET` for replacing values, `DROP` for removing columns, and `AS` for naming intermediates. `|>` must tokenize as a distinct type, not bitwise `|` plus `>`.

  • feature implementation
  • configuration tuning
  • data transformation
  • parser
  • serialization
  • SQL Formatter
  • SQL
  • BigQuery
  • repository patch
  • specialized algorithmic or domain reasoning

Published evidence

Model × harness outcomes

Counts are tied to these exact configurations. Harnesses are not treated as equivalent across benchmarks.

Models
13
Configurations
41
Trials
164

Showing 41 of 41 configurations

ModelHarnessEffortResultPasses
claude-fable-5vertex_aimini-swe-agentpublished configurationlowAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmediumAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationhighAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationxhighAll passed4/4
claude-fable-5vertex_aimini-swe-agentpublished configurationmaxAll passed4/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationlowAll passed4/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmediumMixed3/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationhighAll passed4/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationxhighAll passed4/4
claude-opus-4-8anthropicmini-swe-agentpublished configurationmaxAll passed4/4
claude-sonnet-4-6anthropicmini-swe-agentpublished configurationhighMixed3/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationlowMixed3/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmediumAll passed4/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationhighAll passed4/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationxhighAll passed4/4
claude-sonnet-5anthropicmini-swe-agentpublished configurationmaxAll passed4/4
gemini-3-1-pro-previewvertex_aimini-swe-agentpublished configurationhighMixed2/4
gemini-3-5-flashvertex_aimini-swe-agentpublished configurationmediumMixed3/4
glm-5-2zaimini-swe-agentpublished configurationhighMixed3/4
glm-5-2zaimini-swe-agentpublished configurationmaxAll passed4/4
gpt-5-4openaimini-swe-agentpublished configurationxhighMixed3/4
gpt-5-5openaimini-swe-agentpublished configurationlowMixed1/4
gpt-5-5openaimini-swe-agentpublished configurationmediumAll passed4/4
gpt-5-5openaimini-swe-agentpublished configurationhighMixed3/4
gpt-5-5openaimini-swe-agentpublished configurationxhighAll passed4/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationlowNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmediumNo passes0/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationhighMixed3/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationxhighMixed3/4
gpt-5-6-lunaopenaimini-swe-agentpublished configurationmaxAll passed4/4
gpt-5-6-solopenaimini-swe-agentpublished configurationlowMixed3/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmediumAll passed4/4
gpt-5-6-solopenaimini-swe-agentpublished configurationhighAll passed4/4
gpt-5-6-solopenaimini-swe-agentpublished configurationxhighAll passed4/4
gpt-5-6-solopenaimini-swe-agentpublished configurationmaxAll passed4/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationlowMixed1/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmediumMixed2/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationhighMixed2/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationxhighAll passed4/4
gpt-5-6-terraopenaimini-swe-agentpublished configurationmaxAll passed4/4
kimi-k2-7-codeopenroutermini-swe-agentpublished configurationdefaultMixed1/4