Redirect All Output to File in Bash: Why Some Output Still Shows Up & How to Fix It

Table of Contents#

1. Understanding Bash Output Streams: Stdout, Stderr, and Beyond
2. Common Redirection Methods: What Most Users Try
3. The Mystery: Why Some Output Still Escapes Redirection
4. How to Fix It: Solutions for Every Scenario
5. Advanced Scenarios: Scripts, Subshells, and Systemd Services
6. Conclusion
7. References

Understanding Bash Output Streams: Stdout, Stderr, and Beyond#

Before diving into redirection quirks, let’s clarify how Bash handles output. Every process in Unix-like systems has three standard I/O streams by default:

Key Point: Stdout and Stderr Are Separate#

By default, both stdout (FD 1) and stderr (FD 2) print to the terminal, but they are distinct streams. This separation lets you handle “normal” output and errors differently (e.g., log errors to a file but show stdout on the terminal).

Example:

# stdout (FD 1)
echo "This is normal output"  
 
# stderr (FD 2)
echo "This is an error message" >&2  # Force output to FD 2  

Run this, and both lines appear in the terminal—but under the hood, they’re traveling through separate pipes.

Common Redirection Methods: What Most Users Try#

Most guides teach basic redirection syntax, but let’s recap the essentials to set the stage for our problem.

1. Redirect Stdout Only (> or >>)#

• command > file: Overwrites file with stdout (FD 1).
• command >> file: Appends stdout to file (avoids overwriting).

Example:

echo "Hello, stdout!" > output.log  # stdout → output.log  
cat output.log  # Prints "Hello, stdout!"  

2. Redirect Stderr Only (2> or 2>>)#

To redirect only errors (FD 2), use 2>:

ls missingfile 2> errors.log  # stderr → errors.log  
cat errors.log  # Prints "ls: cannot access 'missingfile': No such file or directory"  

3. Redirect Both Stdout and Stderr#

The most common “catch-all” redirection is > file 2>&1, which redirects stdout to file, then redirects stderr (FD 2) to the same place as stdout (FD 1).

Example:

# Generate both stdout and stderr  
{ echo "Normal output"; echo "Error!" >&2; } > combined.log 2>&1  
 
cat combined.log  # Contains both lines:  
# Normal output  
# Error!  

Bash 4+ also supports the shortcut &> file (or &>> file to append), which is equivalent to > file 2>&1.

So… Why Isn’t This Enough?#

If > file 2>&1 redirects both streams, why do some outputs still appear in the terminal? Let’s uncover the culprits.

The Mystery: Why Some Output Still Escapes Redirection#

Despite redirecting stdout and stderr, output can leak to the terminal due to non-standard streams or redirection misconfiguration. Here are the top causes:

1. Output to /dev/tty (Direct Terminal Access)#

Some programs bypass stdout/stderr entirely and write directly to the terminal via /dev/tty (a special file representing the current terminal). This is common in interactive tools (e.g., password prompts, progress bars) or poorly written scripts that hardcode terminal output.

Example:
A script that forces output to the terminal:

# tty-output.sh  
echo "This goes to stdout"  
echo "This goes to the terminal directly" > /dev/tty  # Bypasses stdout/stderr!  

Now try redirecting:

bash tty-output.sh > output.log 2>&1  

Result:

• output.log contains "This goes to stdout".
• The terminal still shows "This goes to the terminal directly".

2. Unredirected Subshells or Background Processes#

If your command spawns subshells ((...)) or background processes (&), redirection applied to the parent shell may not affect them.

Example:

# Parent shell redirects, but subshell does not inherit properly  
(echo "Subshell output") > parent.log 2>&1 &  # Background subshell  

Here, the subshell runs in the background, and its output may not be captured by parent.log (depending on timing and shell behavior).

3. Buffering Issues#

Bash buffers output for efficiency:

• Line-buffered: Output is flushed when a newline (\n) is encountered (default for stdout when connected to a terminal).
• Block-buffered: Output is flushed only when the buffer is full (default for stdout when redirected to a file).

This can make it seem like output is missing (e.g., a script with echo -n "Processing..." may not write to the log until the buffer flushes, leading you to think redirection failed).

4. Third-Party Tools Logging Directly to the Terminal#

Some tools (e.g., systemd, docker, or custom binaries) use libraries that log directly to the terminal via low-level system calls (e.g., write(1, ...) vs. fprintf(stdout, ...)), bypassing standard streams.

How to Fix It: Solutions for Every Scenario#

Let’s tackle each cause with actionable fixes.

Fix 1: Capture /dev/tty Output with script#

The script command records all terminal activity, including output to /dev/tty. It creates a typescript of the session, capturing even direct terminal writes.

Usage:

script -c "bash tty-output.sh" capture.log  # Run command in script session  

• -c "command": Executes command in the script session.
• capture.log: Output file containing all terminal activity.

Result: capture.log now includes both the stdout output and the /dev/tty line!

Fix 2: Redirect Subshells and Background Processes Explicitly#

Ensure redirection is applied inside subshells or to the entire command group.

Example 1: Redirect the entire subshell

# Apply redirection to the subshell itself  
( echo "Subshell output" ) > subshell.log 2>&1  

Example 2: Redirect background processes

# Redirect output *before* sending to background  
{ echo "Background task"; sleep 2; echo "Done"; } > bg.log 2>&1 &  

Fix 3: Disable Buffering with stdbuf or unbuffer#

To force immediate output (useful for real-time logging), disable buffering with:

stdbuf (Standard Buffering Control)#

• stdbuf -oL command: Line-buffer stdout (flush on newline).
• stdbuf -o0 command: Disable buffering entirely (flush immediately).

Example:

stdbuf -o0 ./long-running-script.sh > output.log 2>&1  

unbuffer (From expect Package)#

The unbuffer command (part of the expect package) makes a command think it’s running in a terminal, forcing line buffering:

unbuffer ./long-running-script.sh > output.log 2>&1  

Fix 4: Identify and Redirect Hidden Streams with strace#

If output still leaks, use strace to trace system calls and see where the program is writing.

Example:

strace -e write bash tty-output.sh 2>&1 | grep write  

Look for lines like:

write(1, "This goes to stdout\n", 19)    = 19  
write(3, "This goes to the terminal directly\n", 33)  # FD 3 = /dev/tty!  

Here, FD 3 is /dev/tty. To capture this, use script (as in Fix 1) or redirect FD 3 explicitly (advanced):

bash tty-output.sh 3>&1 > output.log 2>&1  # Redirect FD 3 to stdout, then capture all  

Advanced Scenarios#

Redirect All Output in a Script#

To ensure all output from a script is redirected (including subshells and background processes), start the script with:

#!/bin/bash  
exec > /path/to/script.log 2>&1  # Redirect all subsequent output  
# Rest of script...  

The exec command replaces the current shell process and applies redirection to all child processes.

Systemd Services: Redirect Output to Journal or File#

Systemd services often log to the journal by default, but you can override this in the service file (/etc/systemd/system/myservice.service):

[Service]  
ExecStart=/path/to/script.sh  
StandardOutput=file:/var/log/myservice.log  
StandardError=file:/var/log/myservice.err.log  

Reload systemd with systemctl daemon-reload and restart the service.

Combining Redirection with tee (Log and Print)#

To redirect output to a file and show it in the terminal, use tee with stderr redirected to stdout:

command 2>&1 | tee output.log  # Both stdout/stderr → output.log and terminal  

Conclusion#

Redirecting all output in Bash看似简单，但隐藏的陷阱（如 /dev/tty 写入、子shell未重定向或缓冲问题）可能导致输出“逃逸”。通过理解标准流、使用 script 捕获终端输出、显式重定向子进程，并调整缓冲，你可以确保所有输出都被正确捕获。

记住：

• 总是测试重定向以验证是否捕获了所有内容。
• 当基础重定向失败时，script 和 strace 是诊断问题的强大工具。
• 缓冲可能导致“延迟”输出，使用 stdbuf 或 unbuffer 解决实时日志需求。

References#

• GNU Bash Manual: Redirection
• man script
• man stdbuf
• man strace
• Expect’s unbuffer Command