Writing Safe Bash Scripts: set -euo pipefail, trap and ShellCheck

By default, Bash carries on after an error, and treats a misspelled variable as empty. That can turn a small bug into a disaster, so careful scripts turn on stricter rules.

The danger#

This script runs cd to a folder that does not exist, and carries on in the wrong place:

mkdir demo
touch demo/important.txt
cat > danger.sh <<'EOF'
#!/bin/bash
cd /no/such/folder
echo "Now in: $(pwd)"
echo "Would delete: $(ls)"
EOF
cd demo
bash ../danger.sh
cd ..

../danger.sh: line 2: cd: /no/such/folder: No such file or directory
Now in: /home/asha/demo
Would delete: important.txt

If the last line had been rm -rf *, it would have deleted the wrong files.

set -e: stop on errors#

cat > safe.sh <<'EOF'
#!/bin/bash
set -e
cd /no/such/folder
echo "This never runs"
EOF
bash safe.sh
echo "Exit code: $?"

safe.sh: line 3: cd: /no/such/folder: No such file or directory
Exit code: 1

set -u: catch unset variables#

Without -u, a typo silently gives an empty value. This is how rm -rf "$BUILD_DIR/"* becomes rm -rf /*.

cat > unset.sh <<'EOF'
#!/bin/bash
set -u
folder="logs"
echo "Using $foldr"
echo "This never runs"
EOF
bash unset.sh
echo "Exit code: $?"

unset.sh: line 4: foldr: unbound variable
Exit code: 1

set -o pipefail: errors inside pipes#

A pipeline normally reports the status of its last command, which hides earlier failures:

cat /nothing 2> /dev/null | wc -l
echo "Status without pipefail: $?"
set -o pipefail
cat /nothing 2> /dev/null | wc -l
echo "Status with pipefail: $?"
set +o pipefail

0
Status without pipefail: 0
0
Status with pipefail: 1

The standard header#

Start your scripts with these lines:

#!/usr/bin/env bash
set -euo pipefail

-e has exceptions: it does not stop for commands used in if, or before && or ||. To allow one command to fail, add || true:

grep "TODO" notes.txt || true

Cleaning up with trap#

trap runs code when the script exits, even after an error. It is the right way to remove temporary files:

cat > cleanup.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

tmp=$(mktemp)
trap 'rm -f "$tmp"; echo "Cleaned up"' EXIT

echo "Working in a temp file" > "$tmp"
false
echo "This never runs"
EOF
bash cleanup.sh
echo "Exit code: $?"

Cleaned up
Exit code: 1

The script failed at false, but the cleanup still ran. mktemp creates a unique file, which avoids clashes and is safer than a fixed name in /tmp.

Traps can also handle signals such as Ctrl+C: trap 'echo Stopped; exit 130' INT.

Say where it failed#

trap ... ERR runs on failure, and $LINENO gives the line:

cat > where.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
trap 'echo "Failed on line $LINENO" >&2' ERR
echo "Step one"
ls /nothing
echo "Step two"
EOF
bash where.sh 2>&1 | grep -v 'cannot access'

Step one
Failed on line 5

Logging#

Add timestamps, and send messages to stderr, so that normal output stays clean:

log() {
  printf '%s %s\n' "$(date +%H:%M)" "$*" >&2
}
log "Backup started" 2>&1 | sed 's/^[0-9:]* /HH:MM /'

HH:MM Backup started

Stop two copies running#

flock makes sure that only one copy runs at once:

exec 9> /tmp/myscript.lock
flock -n 9 || { echo "Already running"; exit 1; }

ShellCheck#

ShellCheck finds bugs in shell scripts. Install it with sudo apt install shellcheck, or paste a script into the website. Use it on every script:

cat > bad.sh <<'EOF'
#!/bin/bash
file="my report.txt"
rm $file
EOF
shellcheck -f gcc bad.sh | cut -d: -f2-

3:4: note: Double quote to prevent globbing and word splitting. [SC2086]

A checklist#

• Start with set -euo pipefail.
• Quote every variable.
• Check arguments, and print a usage message.
• Use mktemp and trap for temporary files.
• Use "$HOME" or full paths, not relative ones.
• Add a --dry-run option for scripts that delete things.
• Run ShellCheck.

Common mistakes#

Practice#

cat > tmpdir.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
dir=$(mktemp -d)
trap 'rm -rf "$dir"' EXIT
touch "$dir/work.txt"
echo "$dir" > last_dir.txt
echo "Files: $(ls "$dir" | wc -l)"
EOF
bash tmpdir.sh
[[ -d "$(cat last_dir.txt)" ]] && echo "Still there" || echo "Removed"

Files: 1
Removed

Key takeaway#

Next: Project: Backup Script.