Shells, pipes and precedence

A fun distraction occurred this week when the following snippet of bash code was suggested in a pull request (simplified for demonstrative clarity):

1if [ echo $VAR1 | grep -q "text" ] || [ $VAR2 -eq 0 ]; then
2   echo "If test passed."
3else
4   echo "If test failed."
5fi   

Taken at initial face value, it looked like it should work okay. There didn’t appear to be anything immediately wrong with its syntax. But upon execution, it resulted in a syntax error:

1-bash: [: missing `]
2grep: ]: No such file or directory

So what’s going on here? After all, the following is quite valid:

1if [ true ] || [ false ]; then
2   echo "If test passed."
3fi

Well, as you’ve probably guessed from comparing the two examples (if not the header of this very article), the difference is the presence of the pipeline.

When we’re talking sh (shell) grammar, the if conditional block belongs to a category known as compound commands. Pipelines, as it turns out, take higher precedence in parsing order than compound commands, and this subsequently affects how bash interprets the whole statement.

If pipelines do take precedence, then bash will treat

if [ echo $VAR1

as a command, and

grep -q "text" ] || [ $VAR2 -eq 0 ]; then ...

as a command. Now that we know this, the errors make more sense. The leading "missing ]" error is because the first command does indeed have a missing closing bracket. The second "grep: ]: No such file or directory" error is because bash is passing grep the "]" as an argument.

So where can we find out more about the way shell grammar is parsed? The authoritative list is the Shell Grammar Rules documentation. Looking specifically at pipelines, we see:

• A pipeline can be made up of pipe_sequences.

pipeline         :      pipe_sequence

• A pipe_sequence can be made up of a command, or more pipe_sequences.

pipe_sequence    :                             command
                 | pipe_sequence '|' linebreak command

• A command can be made up of compound_commands.

command          : compound_command

• And a compound_command can be made up of if_clauses.

compound_command : ...
                 | if_clause

So back to the original topic, how should we fix the suggestion? Well, there’s probably many ways. I would opt to simplify the if conditional and determine the result of the grep prior:

1VAR1_TEST=$(echo $VAR1 | grep -q "test")
2VAR1_RC=$?
3if [ $VAR1_RC -eq 0 ] || [ $VAR2 -eq 0 ]; then
4...

However, in this specific case, we could also make use of the bash =~ operator to apply a regex test against $VAR1 instead.

1if [[ $VAR1 =~ "test" ]] || [ $VAR2 -eq 0 ]; then
2...

Now you might be wondering - why did we use [[ and ]] here, and why not single brackets? Well, using the double-brackets enables more functionality - including the very =~ operator used in this test. There’s a good write-up of the differences on this site.