Developing Portable Bash Scripts for Different Unix Systems

Table of Contents

1. Understanding Unix System Variations
2. Key Principles for Portable Bash Scripts
3. Critical Technical Considerations
4. Practical Example: A Portable Script
5. Testing Across Unix Systems
6. Common Pitfalls and How to Avoid Them
7. Best Practices for Maintainable Portable Scripts
8. Conclusion
9. References

1. Understanding Unix System Variations

To write portable scripts, you first need to recognize the key differences between Unix systems. Here are the most impactful:

1.1 GNU vs. BSD Utilities

Many core Unix utilities (e.g., sed, awk, grep, ls) have two dominant implementations:

• GNU Coreutils: Used in Linux distributions (Ubuntu, Fedora, Arch) and provide extended features (e.g., sed -i without backups, grep --color).
• BSD Utilities: Used in macOS, FreeBSD, and OpenBSD. They prioritize POSIX compliance but often omit GNU-specific flags and may require different syntax (e.g., sed -i '' for in-place edits, as BSD sed requires a backup suffix argument).

1.2 File System Layouts

Path conventions vary:

• Linux: Tools like bash or python are often in /bin or /usr/bin.
• macOS: Some system tools live in /usr/bin, but user-installed tools (via Homebrew) go to /usr/local/bin.
• FreeBSD: Base utilities are in /bin or /usr/bin, while ports/packages use /usr/local/bin.
  Hard-coding paths like /usr/local/bin/python will break on systems where Python lives elsewhere.

1.3 Default Shells

The default system shell (/bin/sh) differs:

• Linux (Debian/Ubuntu): /bin/sh links to dash (a minimal POSIX shell, not bash).
• Linux (Fedora/RHEL): /bin/sh links to bash (but runs in POSIX mode).
• macOS: /bin/sh links to bash (version 3.2, due to licensing constraints).
• FreeBSD: /bin/sh is the native sh (a POSIX-compliant shell, not bash).
  Scripts relying on bash-specific features (e.g., arrays, [[ ]] conditionals) will fail if run with dash or BSD sh.

2. Key Principles for Portable Bash Scripts

Portability starts with intentional design. Follow these core principles:

2.1 Use POSIX-Compliant Syntax

Stick to the POSIX shell standard, which all Unix shells (including dash, BSD sh, and bash in POSIX mode) support. Avoid bashisms like:

• Arrays (myarray=(a b c)).
• Extended globbing (shopt -s extglob).
• Process substitution (<(command)).
• [[ ]] conditionals (use [ ] instead).

2.2 Avoid Hard-Coded Paths

Never assume tools live in specific locations. Instead:

• Use command -v to check if a tool exists (e.g., if ! command -v curl >/dev/null; then echo "curl not found"; exit 1; fi).
• Rely on $PATH to resolve executables (e.g., just use curl instead of /usr/bin/curl).

2.3 Explicitly Check for Dependencies

Always verify that required tools (e.g., sed, awk, git) are installed before running. A script that fails because jq is missing is better than one that crashes with a cryptic “command not found” error.

2.4 Handle Command Variations Gracefully

When utilities behave differently (e.g., GNU vs. BSD sed), detect the system type and adjust behavior. Use uname -s to identify the OS:

OS=$(uname -s)
if [ "$OS" = "Darwin" ]; then  # macOS
    SED_CMD="sed -i ''"
elif [ "$OS" = "FreeBSD" ]; then  # FreeBSD
    SED_CMD="sed -i ''"
else  # Linux (GNU sed)
    SED_CMD="sed -i"
fi

2.5 Quote Variables and Arguments

Unquoted variables break when they contain spaces or special characters (e.g., file name.txt). Always quote variables:

# Bad: Fails if $filename has spaces
rm $filename  

# Good: Safe for spaces/special chars
rm "$filename"  

3. Critical Technical Considerations

3.1 The Shebang Line: #!/bin/sh vs #!/bin/bash

The shebang (#!) determines the interpreter. For portability:

• Use #!/bin/sh to enforce POSIX compliance. This ensures the script runs in the system’s native POSIX shell (e.g., dash, BSD sh), avoiding bashisms.
• Avoid #!/bin/bash unless your script explicitly requires bash-only features (and document this dependency!).

3.2 Portable Command Substitution: $() vs ` `

Both $(command) and `command` capture command output, but $() is:

• More readable (avoids backtick escaping issues).
• POSIX-compliant (supported by all modern shells).
  Use $() instead of ` `:

# Portable
current_dir=$(pwd)  

# Avoid (harder to read, especially with nested commands)
current_dir=`pwd`  

3.3 Conditionals: [ ] (POSIX) vs [[ ]] (Bash)

[[ ]] is a bash-specific enhancement with features like pattern matching and logical operators (&&, ||). For portability, use the POSIX-compliant [ ] (aka test command) with explicit syntax:

# Bad (bash-specific)
if [[ "$var" == *pattern* ]]; then ...  

# Good (POSIX-compliant)
if [ "$var" = "expected_value" ]; then ...  

Note: With [ ], always:

• Quote variables to avoid word-splitting.
• Use -a for logical AND ([ "$a" -eq 1 -a "$b" -eq 2 ]), -o for OR.
• Avoid =~ (regex matching, bash-specific).

3.4 Handling Utilities with System-Specific Behavior

Example 1: sed -i (In-Place Edits)

GNU sed allows sed -i 's/old/new/' file (no backup), but BSD sed requires a backup suffix (e.g., sed -i '' 's/old/new/' file to disable backups). To handle both:

# Portable sed -i wrapper
sed_inplace() {
    if sed --version 2>/dev/null | grep -q GNU; then
        # GNU sed: no backup suffix needed
        sed -i "$@"
    else
        # BSD sed: require empty backup suffix
        sed -i '' "$@"
    fi
}

# Usage: sed_inplace 's/old/new/' file.txt

Example 2: date Formatting

GNU date supports date -d "2 days ago", but BSD date (macOS/FreeBSD) uses date -v -2d. Use a portable alternative like:

# Portable "2 days ago" (works on GNU/BSD date)
two_days_ago=$(date -j -f "%Y-%m-%d" "$(date +%Y-%m-%d) - 2 days" +%Y-%m-%d 2>/dev/null || date -d "2 days ago" +%Y-%m-%d)

3.5 Portable Path Resolution

Use command -v to check if a tool exists in $PATH before using it:

# Check if curl is available
if ! command -v curl >/dev/null 2>&1; then
    echo "Error: curl is required but not installed." >&2
    exit 1
fi

3.6 Error Handling and Robustness

• set -e: Exit on error (POSIX-compliant, but use cautiously—some commands return non-zero for non-fatal issues).
• set -u: Treat undefined variables as errors (helps catch typos). Note: Older shells (e.g., Solaris sh) may not support set -u, so document this if used.
• trap: Clean up temporary files on exit:

  temp_file=$(mktemp)
  trap 'rm -f "$temp_file"' EXIT  # Delete temp_file on script exit

4. Practical Example: A Portable Script

Here’s a script that demonstrates portability best practices: it checks for dependencies, uses POSIX syntax, and handles sed -i across systems.

#!/bin/sh
# Purpose: Replace "old_text" with "new_text" in a file (portable across Unix)

set -euo pipefail  # Exit on error, undefined var, or pipeline failure

# --------------------------
# Check dependencies
# --------------------------
check_dependency() {
    if ! command -v "$1" >/dev/null 2>&1; then
        echo "Error: Required tool '$1' not found. Please install it first." >&2
        exit 1
    fi
}

check_dependency "sed"
check_dependency "mktemp"  # For safe temp file handling


# --------------------------
# Validate input
# --------------------------
if [ $# -ne 1 ]; then
    echo "Usage: $0 <file-to-edit>" >&2
    exit 1
fi

file="$1"
if [ ! -f "$file" ]; then
    echo "Error: File '$file' does not exist." >&2
    exit 1
fi


# --------------------------
# Portable sed -i wrapper
# --------------------------
sed_inplace() {
    if sed --version 2>/dev/null | grep -q GNU; then
        sed -i "$@"
    else
        sed -i '' "$@"
    fi
}


# --------------------------
# Main logic
# --------------------------
echo "Replacing 'old_text' with 'new_text' in $file..."
sed_inplace 's/old_text/new_text/g' "$file"

echo "Done. Modified file: $file"

5. Testing Across Unix Systems

Portability is meaningless without testing on target systems. Use these tools:

5.1 Local Testing with Containers/VMs

• Docker: Test Linux distros with official images:

  # Test on Ubuntu (GNU utilities)
  docker run --rm -v "$(pwd):/scripts" ubuntu:latest sh /scripts/your_script.sh
  
  # Test on Alpine (musl libc, busybox utilities)
  docker run --rm -v "$(pwd):/scripts" alpine:latest sh /scripts/your_script.sh

• Vagrant: Test FreeBSD or older Linux versions with Vagrant boxes (e.g., freebsd/FreeBSD-13.0-RELEASE).
• macOS: Use a physical Mac or Apple’s Xcode Cloud for CI.

5.2 CI/CD for Automated Testing

Leverage CI services to test across systems:

• GitHub Actions: Test on Ubuntu, macOS, and even FreeBSD (via actions/runner-images).
• GitLab CI: Use ubuntu, fedora, and macos runners.

6. Common Pitfalls and How to Avoid Them

7. Best Practices for Maintainable Portable Scripts

1. Keep It Simple: Avoid over-engineering. Use built-in tools instead of complex dependencies.
2. Lint with ShellCheck: Run ShellCheck to catch non-portable syntax (e.g., shellcheck your_script.sh).
3. Document Dependencies: List required tools (e.g., curl, jq) in a comment or README.
4. Test Early, Test Often: Validate on target systems during development, not just at release.
5. Use set -euo pipefail: Enforce strict error checking (document if set -u is used, as some old shells lack support).

8. Conclusion

Writing portable bash scripts requires mindfulness of Unix variations, adherence to POSIX standards, and rigorous testing. By avoiding bashisms, handling utility differences gracefully, and validating across systems, you can ensure your scripts work reliably on Linux, macOS, FreeBSD, and beyond. The effort pays off in broader usability and fewer “works for me” bugs.

9. References

• POSIX Shell Specification
• GNU Coreutils vs. BSD Utilities
• ShellCheck: Static Analysis for Shell Scripts
• Docker Hub: Official Unix Images
• FreeBSD Porter’s Handbook (for BSD-specific conventions)