Signs of Triviality

Opinions, mostly my own, on the importance of being and other things.
[homepage] [index] [] [@jschauma] [RSS]

Writing Shell Scripts

April 8th, 2016

Spaghetti on the floor90% of shell scripts are crud. Sturgeon's Law notwithstanding, otherwise capable and careful developers create sloppy and fragile scripts. Often times these scripts exhibit characteristics that they would not otherwise approve of.

There may be many reasons for this phenomenon: the mental approach we take ("I'll just throw together a quick script."), the same effect TIMTOWTDI has on perl, a lack of motivation or respect ("It's just a shell script."), and the dangerous efficiency of scripting are likely some. Shell scripts often are supportive glue, duct tape, not the actual product you care about. This easily leads to carelessly slapped together, knee-deep spaghetti code.

Writing shell scripts is easy. Writing clean, reliable, robust, shell scripts is easy, too, but you can't be lazy. Here are a few recommendations to improve your shell scripts.

bash != /bin/sh

Don't assume bash. bash may not be available on the systems your script needs to run on. If it is available, it may not be in /bin/bash, and you cannot assume /bin/sh to be bash (or bash-as-sh).

set -eu

Don't allow your code to barrel on after something failed (set -e). Avoid nasty surprises from fat-fingering a variable name and let the shell bail out when it encounters an unset variable (set -u).

Set a umask

Be explicit in the permissions you need. Avoid race conditions on private files between their creation and you calling chmod(1). Default to umask 077, set a more open umask if needed.

Set the umask right at the beginning of the script.

Use mktemp(1)

Avoid temporary files whenever you can. When you can't, make sure to create them safely. Do not hard-code temporary file names: a file may already exist, and you may not have permissions to overwrite it.

Be careful when using predictable temporary files -- e.g. /tmp/script.$$ -- as those may lead to a race condition. Instead, use mktemp(1). Combined with a restrictive umask, you can then create a safe temporary directory to which only the EUID has access.

(Honor the user's TMPDIR variable; they may not want to use /tmp.)

Much more on safely creating and using temporary files here.

Use an exit handler

Clean up your temporary files (and other resources) upon exit. Use an exit handler to ensure this happens every time your script terminates.

E.g., set trap "cleanup" 0 near the beginning of the script.

Use functions

You wouldn't write all your code in one gigantic "main" in any other language; don't do this in your shell scripts, either. Use properly scoped functions with meaningful return values. Indent your functions and format them to be readable. Refactor if your code runs off the end of the line or scrolls across multiple screen fulls.

Use scoped variables

Use properly scoped variables. If your shell supports it (although not part of POSIX, many shells installed as /bin/sh do), use local variables in your functions, global variables for script-wide settings. Use a naming convention that facilitates easy reading, e.g. UPPER_CASE for global and lower_case for local variables. To distinguish your variables from e.g. environment variables, consider prefixing them with a string like __.

Use readonly for variables that you do not plan on changing.

Quote all variables

Always quote all variables to avoid surprises when a value contains white space. Use braces around all variables to make it obvious which parts of a longer string are part of which variable. Likewise, the use of $() for command substitutions is more readable than the use of backticks (``).

Error messages go to stderr

Don't write error messages to stdout -- redirect them to stderr instead. Don't suppress error messages unless you know for certain that the user has no interest in seeing them.

Read from stdin, write to stdout

Dealing with file I/O is tedious and contains many pitfalls. You can avoid those by following the Unix philosophy: read input from stdin, generate output to stdout.

Let the user deal with redirecting output where they want or need it to go.

Use a meaningful exit status

Signal the user when your script completed successfully by using a meaningful exit status.

Putting everything together, a clean, readable, robust shell script might look like this. (And yes, of course the fictional hfrob command does have a manual page.)

Happy shelling, pull requests welcome, and all that jazz.

April 8th, 2016


[Moving the Needle] [Index] [Root Cause: Human Errno]