Managing Files and Directories with Bash Scripting

Table of Contents

1. Understanding the Basics of Bash Scripting
2. Core File and Directory Operations
   • Creating Directories (mkdir)
   • Creating Files (touch)
   • Copying Files/Directories (cp)
   • Moving/Renaming Files (mv)
   • Deleting Files/Directories (rm)
3. Wildcards and Pattern Matching
4. Loops for Batch Processing
5. Conditional Checks for File/Directory Validation
6. File Permissions and Ownership
7. Practical Example: A File Organizer Script
8. Best Practices for Safe File Management
9. References

1. Understanding the Basics of Bash Scripting

Before diving into file management, let’s cover the fundamentals of bash scripting:

What is a Bash Script?

A bash script is a text file containing a sequence of bash commands. It allows you to automate repetitive tasks by executing multiple commands in order.

Shebang Line

Every bash script should start with the shebang line, which tells the system which interpreter to use:

#!/bin/bash

This line ensures the script runs with bash (not another shell like sh or zsh).

Making Scripts Executable

To run a script, you must first make it executable using chmod:

chmod +x my_script.sh

Then execute it with:

./my_script.sh  # Runs the script in the current directory

2. Core File and Directory Operations

Bash provides built-in commands to manipulate files and directories. Let’s break down the most essential ones.

Creating Directories: mkdir

The mkdir (make directory) command creates new directories.

Syntax:

mkdir [options] directory_name

Common Options:

• -p: Create parent directories if they don’t exist (e.g., mkdir -p docs/reports/2024 creates docs, reports, and 2024).
• -v: Verbose mode (print a message for each directory created).

Examples:

# Create a single directory
mkdir projects

# Create nested directories
mkdir -p projects/bash_scripts/tutorials

# Verbose mode
mkdir -v -p downloads/images

Creating Files: touch

The touch command creates empty files or updates the timestamp of existing files.

Syntax:

touch filename1 filename2 ...

Examples:

# Create a single empty file
touch notes.txt

# Create multiple files
touch report.pdf data.csv image.jpg

# Update timestamp of an existing file (no change to content)
touch old_file.txt

Copying Files/Directories: cp

The cp (copy) command duplicates files or directories.

Syntax:

cp [options] source destination

Common Options:

• -r or -R: Recursively copy directories (required for copying folders).
• -i: Interactive mode (prompt before overwriting existing files).
• -v: Verbose mode (show copied files).

Examples:

# Copy a file to another location
cp notes.txt backup/notes_backup.txt

# Copy a directory and its contents
cp -r projects/bash_scripts projects/bash_scripts_backup

# Interactive copy (avoids accidental overwrites)
cp -i important.docx backup/

Moving/Renaming Files: mv

The mv (move) command moves files/directories to a new location or renames them.

Syntax:

mv [options] source destination

Common Options:

• -i: Prompt before overwriting.
• -v: Verbose mode.

Examples:

# Rename a file
mv report_v1.pdf report_final.pdf

# Move a file to a directory
mv photos/summer.jpg downloads/

# Move and rename a directory
mv old_projects/ archived_projects/

Deleting Files/Directories: rm

The rm (remove) command deletes files and directories. Use with caution—deleted files are not sent to the trash!

Syntax:

rm [options] file_or_directory

Common Options:

• -i: Interactive mode (prompt before deletion).
• -r or -R: Recursively delete directories and their contents.
• -f: Force deletion (ignore non-existent files, no prompts).

Dangers of rm -rf:
The command rm -rf directory/ deletes directory and all its contents recursively without prompts. Accidental use (e.g., rm -rf / or rm -rf *) can cause catastrophic data loss. Always double-check paths!

Safe Examples:

# Delete a single file (interactive)
rm -i notes.txt

# Delete an empty directory
rmdir old_folder  # Only works for empty dirs

# Delete a non-empty directory (use -r with caution!)
rm -r -i projects/old_tutorials  # -i prompts for each file

3. Wildcards and Pattern Matching

Wildcards (also called glob patterns) let you match multiple files/directories using special characters. They’re critical for batch operations.

Examples:

# Delete all .tmp files
rm *.tmp

# Copy all .jpg and .png files to a directory
cp *.{jpg,png} images/

# Match files starting with "data_" and ending with ".csv"
ls data_*.csv

# Match files with exactly 3 characters (e.g., "abc.txt", "123.pdf")
ls ???.txt

4. Loops for Batch Processing

Loops let you automate repetitive tasks, like processing multiple files. The most common loop in bash is the for loop.

Basic for Loop Syntax

for item in list_of_items; do
  # Commands to run for each item
done

Examples:

Example 1: Rename All .txt Files

Add a prefix (e.g., backup_) to all .txt files:

for file in *.txt; do
  mv "$file" "backup_$file"
done

Example 2: Compress All .log Files

Zip each .log file into a .zip archive:

for logfile in *.log; do
  zip "${logfile%.log}.zip" "$logfile"  # Remove .log extension for zip name
done

Example 3: Process Files in Subdirectories

Loop through all .pdf files in docs/ and its subdirectories:

# Use find to list all PDFs recursively
for pdf in $(find docs/ -name "*.pdf"); do
  echo "Processing: $pdf"
  # Add commands here (e.g., copy to a central folder)
done

5. Conditional Checks for File/Directory Validation

Conditionals (if statements) let you check if a file/directory exists, is readable, or meets other criteria before performing actions.

File/Directory Test Operators

Use these operators in if statements to validate files/directories:

Example Scripts

Example 1: Check if a File Exists Before Deleting

file="old_data.csv"

if [ -f "$file" ]; then
  echo "Deleting $file..."
  rm "$file"
else
  echo "$file does not exist. Nothing to delete."
fi

Example 2: Create a Directory if It Doesn’t Exist

dir="new_project"

if [ ! -d "$dir" ]; then  # "!" negates the condition
  echo "Creating directory: $dir"
  mkdir "$dir"
else
  echo "Directory $dir already exists."
fi

Example 3: Check if a File is Empty

log_file="app.log"

if [ -s "$log_file" ]; then
  echo "$log_file has content. Archiving..."
  gzip "$log_file"
else
  echo "$log_file is empty. Deleting..."
  rm "$log_file"
fi

6. File Permissions and Ownership

Bash scripts often need to handle file permissions (who can read, write, or execute a file) and ownership (which user/group owns the file).

Viewing Permissions: ls -l

Use ls -l to see permissions for files/directories:

ls -l notes.txt
# Output: -rw-r--r-- 1 user group 1024 Jun 1 12:00 notes.txt

The first 10 characters (-rw-r--r--) represent permissions:

• 1st: File type (- for regular file, d for directory).
• Next 3: Owner permissions (rw- = read/write).
• Next 3: Group permissions (r-- = read-only).
• Next 3: Others permissions (r-- = read-only).

Changing Permissions: chmod

The chmod (change mode) command modifies permissions. Use numeric mode (e.g., 755) or symbolic mode (e.g., u+x).

Numeric Mode

Permissions are represented by 3 digits (owner, group, others), where each digit is a sum of:

• 4: Read (r)
• 2: Write (w)
• 1: Execute (x)

Common Numeric Permissions:

• 755: Owner can read/write/execute; group/others can read/execute (good for scripts).
• 644: Owner read/write; group/others read-only (good for documents).

Example:

# Make a script executable (owner: rwx, group/others: rx)
chmod 755 my_script.sh

# Restrict a sensitive file to owner-only access
chmod 600 secrets.txt

Symbolic Mode

Symbolic mode uses letters to specify changes:

• u: Owner
• g: Group
• o: Others
• a: All (u + g + o)
• +: Add permission
• -: Remove permission
• =: Set exact permission

Example:

# Add execute permission for the owner
chmod u+x my_script.sh

# Remove write permission for others
chmod o-w public_notes.txt

# Give read access to all
chmod a+r report.pdf

Changing Ownership: chown

The chown (change owner) command sets the owner/group of a file/directory (requires sudo for system files).

Syntax:

chown [owner]:[group] file

Example:

# Change owner to "user" and group to "staff"
sudo chown user:staff important_data.csv

7. Practical Example: A File Organizer Script

Let’s build a script that organizes files in a directory into subdirectories (e.g., Images, Documents, Videos) based on their extensions.

Script: organize_files.sh

#!/bin/bash

# Define the target directory (use current directory if not specified)
TARGET_DIR="${1:-.}"  # $1 is the first argument; default to . (current dir)

# Create subdirectories if they don't exist
mkdir -p "$TARGET_DIR"/{Images,Documents,Videos,Music,Archives,Others}

# Loop through all files in the target directory
for file in "$TARGET_DIR"/*; do
  # Skip directories (we only process files)
  [ -d "$file" ] && continue

  # Get the file extension (convert to lowercase)
  ext="${file##*.}"  # Extracts everything after the last "."
  ext=$(echo "$ext" | tr '[:upper:]' '[:lower:]')  # Lowercase

  # Move file to the appropriate directory
  case "$ext" in
    jpg|jpeg|png|gif|bmp)
      mv -v "$file" "$TARGET_DIR/Images/"
      ;;
    doc|docx|pdf|txt|csv|xlsx|pptx)
      mv -v "$file" "$TARGET_DIR/Documents/"
      ;;
    mp4|mov|avi|mkv)
      mv -v "$file" "$TARGET_DIR/Videos/"
      ;;
    mp3|wav|flac)
      mv -v "$file" "$TARGET_DIR/Music/"
      ;;
    zip|tar|gz|rar)
      mv -v "$file" "$TARGET_DIR/Archives/"
      ;;
    *)
      # Unknown extension: move to Others
      mv -v "$file" "$TARGET_DIR/Others/"
      ;;
  esac
done

echo "File organization complete!"

How to Use the Script

1. Save it as organize_files.sh.
2. Make it executable: chmod +x organize_files.sh.
3. Run it (organize current directory by default):

   ./organize_files.sh

   Or specify a target directory:

   ./organize_files.sh ~/Downloads

8. Best Practices for Safe File Management

File operations can be destructive. Follow these tips to avoid data loss:

1. Backup First: Always back up files before running scripts that modify/delete data.
2. Test with echo: Before using mv or rm, test with echo to preview changes:

   # Instead of: mv *.txt backup/
   echo mv *.txt backup/  # Shows what would be moved

3. Use -i for Critical Operations: Add -i to cp, mv, or rm to prompt before overwriting/deleting.
4. Avoid rm -rf in Scripts: Never use rm -rf with variables or wildcards (e.g., rm -rf $DIR/*). A typo could delete unintended files.
5. Handle Errors: Use set -e at the top of scripts to exit immediately if any command fails:

   #!/bin/bash
   set -e  # Exit on error

6. Quote Variables: Always enclose variables in quotes (e.g., "$file") to handle filenames with spaces:

   # Bad: mv $file backup/ (fails if $file has spaces)
   # Good: mv "$file" backup/

9. References

• GNU Bash Manual
• GNU Coreutils (mkdir, cp, mv, rm, etc.)
• TLDR Pages (Simplified command documentation)
• Bash Scripting Guide (Cyberciti.biz)
• Book: Learning the Bash Shell by Cameron Newham (O’Reilly)

By mastering these tools and techniques, you’ll be able to automate file management tasks efficiently and safely. Happy scripting! 🚀