Shells, pipes and precedence
A fun distraction occurred this week when the following snippet of
bash code was suggested in a pull request (simplified for demonstrative clarity):
1if [ echo $VAR1 | grep -q "text" ] || [ $VAR2 -eq 0 ]; then 2 echo "If test passed." 3else 4 echo "If test failed." 5fi
Taken at initial face value, it looked like it should work okay. There didn’t appear to be anything immediately wrong with its syntax. But upon execution, it resulted in a syntax error:
1-bash: [: missing `] 2grep: ]: No such file or directory
So what’s going on here? After all, the following is quite valid:
1if [ true ] || [ false ]; then 2 echo "If test passed." 3fi
Well, as you’ve probably guessed from comparing the two examples (if not the header of this very article), the difference is the presence of the pipeline.
When we’re talking
sh (shell) grammar, the
if conditional block belongs to a category known as
compound commands. Pipelines, as it turns out, take higher precedence in parsing order than compound commands, and this subsequently affects how
bash interprets the whole statement.
If pipelines do take precedence, then
bash will treat
if [ echo $VAR1
as a command, and
grep -q "text" ] || [ $VAR2 -eq 0 ]; then ...
as a command. Now that we know this, the errors make more sense. The leading
"missing ]" error is because the first command does indeed have a missing closing bracket. The second
"grep: ]: No such file or directory" error is because
bash is passing
"]" as an argument.
So where can we find out more about the way shell grammar is parsed? The authoritative list is the Shell Grammar Rules documentation. Looking specifically at pipelines, we see:
pipelinecan be made up of
pipeline : pipe_sequence
pipe_sequencecan be made up of a
command, or more
pipe_sequence : command | pipe_sequence '|' linebreak command
commandcan be made up of
command : compound_command
- And a
compound_commandcan be made up of
compound_command : ... | if_clause
So back to the original topic, how should we fix the suggestion? Well, there’s probably many ways. I would opt to simplify the
if conditional and determine the result of the
1VAR1_TEST=$(echo $VAR1 | grep -q "test") 2VAR1_RC=$? 3if [ $VAR1_RC -eq 0 ] || [ $VAR2 -eq 0 ]; then 4...
However, in this specific case, we could also make use of the bash
=~ operator to apply a regex test against
1if [[ $VAR1 =~ "test" ]] || [ $VAR2 -eq 0 ]; then 2...
Now you might be wondering - why did we use
]] here, and why not single brackets? Well, using the double-brackets enables more functionality - including the very
=~ operator used in this test. There’s a good write-up of the differences on this site.