Search across branches using git grep on a specific ref
Tests ability to search across branches using git grep on a specific ref. Evaluates revision-scoped search.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
echo 'version = 1.0' > version.txt - 05
git add version.txt - 06
git commit -m 'Initial version 1.0' - 07
git checkout -b feature/v2 - 08
echo 'version = 2.0' > version.txt - 09
git add version.txt - 10
git commit -m 'Bump to version 2.0' - 11
git checkout main - 12
echo 'git grep version feature/v2' > .grep_command - 13
git add .grep_command - 14
git commit -m 'Add grep sentinel'
Prompt
Here is the output of a git grep command run against the feature/v2 branch. What version number is found? Output ONLY the version number, nothing else.
Expected
2.0
Raw Attempt Evidence
Loading raw attempt evidence…
Model Outputs (14)
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}