Bash Input and Output

Standard File Descriptors

Every Unix process starts with three open file descriptors. Understanding them is the foundation of redirection — you cannot reason about where output goes or why error messages appear in unexpected places without knowing what these three channels are and how they relate.

echo "normal output"          # goes to stdout (fd 1)
echo "error message" >&2      # goes to stderr (fd 2) — keeps errors separate from data
read -r input                 # reads from stdin (fd 0)

echo vs printf

echo works for quick output but has inconsistent behavior across platforms — particularly with -n and escape sequences. printf is predictable, supports format strings, and is the right choice for scripts that need aligned columns, zero-padded numbers, or consistent output across systems.

# echo — simple output
echo "Hello"
echo -n "No newline"
echo -e "Tab:\there\nNewline"

# printf — predictable format strings (prefer in scripts)
printf "Hello\n"
printf "Name: %-10s Age: %d\n" "Alice" 30   # left-aligned, padded to 10 chars
printf "Price: $%.2f\n" 19.99
printf "%05d\n" 42          # 00042 — zero-padded to 5 digits

printf format specifiers for common tasks:

printf "%-20s %s\n" "Key" "Value"        # left-aligned, padded to 20 chars
printf "%20s %s\n"  "Key" "Value"        # right-aligned
printf "0x%X\n" 255                      # 0xFF  (uppercase hex)
printf "%b\n" "line1\nline2"             # interprets backslash escapes

Reading Input

The read command reads a line from stdin into a variable. It is how scripts accept user input interactively, and also how they process data from files and pipes line by line. The -r flag is almost always correct — without it, backslashes in the input are interpreted as escape characters.

# Basic read from stdin
read -r name
echo "Hello, $name"

# With a prompt (no separate echo needed)
read -rp "Enter your name: " name

# Silent input — for passwords (characters are not echoed)
read -rsp "Password: " password
echo ""   # print a newline after the silent input

# With a timeout — proceed with a default if no input arrives
if read -rt 10 -p "Continue? [y/N]: " answer; then
  echo "Got: $answer"
else
  echo "Timed out — using default"
fi

# Read multiple values from a single line
read -r first last <<< "Alice Smith"
echo "First: $first, Last: $last"

# Read into an array (splits on IFS)
read -ra parts <<< "one two three"
echo "${parts[1]}"   # two

Output Redirection

Redirection lets you send output to files instead of the terminal, which is essential for logging, capturing results, and separating normal output from errors. The key insight is that redirections are processed left to right — order matters.

# Redirect stdout to file (overwrite)
echo "output" > file.txt

# Append stdout to file
echo "more output" >> file.txt

# Redirect stderr to file
command 2> errors.log

# Redirect both stdout and stderr to the same file
command > all.log 2>&1
command &> all.log          # bash shorthand — identical result

# Redirect stderr into the stdout stream (merge errors into the pipeline)
command 2>&1 | grep "ERROR"

# Suppress output entirely
command > /dev/null          # discard stdout
command 2> /dev/null         # discard stderr
command &> /dev/null         # discard everything

# Write an error message to stderr (the correct place for error output)
echo "ERROR: something failed" >&2

Input Redirection

Input redirection feeds data to a command’s stdin from a file, a string, or a heredoc — without the command needing to know where the data comes from.

# Feed a file to command's stdin
sort < names.txt

# Feed a string directly to stdin with a herestring (no subshell)
wc -w <<< "hello world foo bar"    # 4

# Heredoc as stdin — useful for multi-line input to commands
mysql -u root mydb <<EOF
SELECT * FROM users LIMIT 10;
EOF

Pipes

Pipes are the mechanism that makes Unix tools composable. Each command in a pipeline reads from stdin and writes to stdout, so you can chain any number of tools together to build a data transformation pipeline. The power comes from combining small, focused tools rather than writing monolithic programs.

# Basic pipe
ls -la | grep ".sh"

# Multi-stage pipeline — find the top 10 attacking IPs in an auth log
cat /var/log/auth.log \
  | grep "Failed password" \
  | awk '{print $11}' \
  | sort | uniq -c \
  | sort -rn \
  | head -10

# Pipe stderr through the pipeline too
command 2>&1 | tee output.log

# Named pipe (FIFO) for inter-process communication
mkfifo /tmp/mypipe
tail -f /var/log/app.log > /tmp/mypipe &
grep --line-buffered "ERROR" < /tmp/mypipe

tee — Split Output

tee solves a specific problem: you want to see output in real time on the terminal AND save it to a file. Without tee, you have to choose one or the other. In deployment and CI scripts, always pipe through tee so you get both visibility and a permanent record.

# Display output AND save to file simultaneously
make build 2>&1 | tee build.log

# Append to an existing log file
./run_tests.sh | tee -a test_results.log

# Write to multiple destinations at once
echo "important message" | tee /var/log/app.log /tmp/debug.log

# Use in a pipeline without interrupting the flow
cat data.txt | tee original.txt | sort | tee sorted.txt | uniq > unique.txt

A standard deploy logging pattern that captures everything to a timestamped log file:

deploy() {
  local logfile="/var/log/deploy/$(date +%Y%m%d_%H%M%S).log"
  mkdir -p "$(dirname "$logfile")"

  {
    echo "=== Deploy started at $(date) ==="
    run_migrations
    restart_services
    run_smoke_tests
    echo "=== Deploy finished at $(date) ==="
  } 2>&1 | tee "$logfile"   # all output goes to terminal and log file
}

Process Substitution

Process substitution <(command) creates a temporary file descriptor that delivers command output as if it were a file. Its most important use is avoiding the subshell that a pipe creates — when a while loop reads from a pipe, variable changes inside the loop are lost when the pipe’s subshell exits.

# Compare the output of two commands as if they were files
diff <(sort file1.txt) <(sort file2.txt)

# Loop over command output WITHOUT a subshell
while IFS= read -r line; do
  echo "Processing: $line"
done < <(find /var/log -name "*.log" -mtime +7)

# Pass command output as a file argument
wc -l < <(grep "ERROR" /var/log/app.log)

The subshell problem illustrated:

# WRONG — count is modified in a subshell created by the pipe, then lost
count=0
grep "ERROR" log.txt | while IFS= read -r line; do
  (( count++ ))
done
echo "$count"   # Always 0 — the subshell's changes did not propagate back

# CORRECT — process substitution keeps the loop in the current shell
count=0
while IFS= read -r line; do
  (( count++ ))
done < <(grep "ERROR" log.txt)
echo "$count"   # Correct value

File Descriptor Tricks

Custom file descriptors let you read from and write to multiple streams in the same script without them interfering with each other. This is useful for scripts that need to process two files simultaneously or redirect all output globally.

# Open a file for reading on a custom file descriptor (3)
exec 3< /etc/hosts
while IFS= read -r line <&3; do
  echo "$line"
done
exec 3<&-   # close FD 3 when done

# Open a file for writing on a custom file descriptor (4)
exec 4> /tmp/output.txt
echo "Line 1" >&4
echo "Line 2" >&4
exec 4>&-   # close FD 4

# Redirect ALL output from this script to a log file (affects everything below)
exec > >(tee -a /var/log/myscript.log) 2>&1
echo "This goes to both terminal and log file"

Capturing Exit Code After a Pipe

In a pipeline, $? gives only the exit code of the last command. PIPESTATUS is a Bash array holding the exit codes of all commands in the most recent pipeline.

grep "pattern" file.txt | sort | uniq

echo "${PIPESTATUS[@]}"    # e.g. "0 0 0" — all three succeeded
echo "${PIPESTATUS[0]}"    # grep's exit code (1 if no match found)
echo "${PIPESTATUS[1]}"    # sort's exit code

# With set -o pipefail, the pipeline's exit code is the first non-zero stage
set -o pipefail
grep "pattern" /nonexistent | sort
echo $?   # non-zero (grep failed)

What’s Next

The next tutorial covers text processing — grep, sed, awk, cut, sort, uniq, wc, tr, and column for working with structured and unstructured text.