Linux Performance Tuning: Common Mistakes and How to Avoid Them

Table of Contents

1. Introduction
2. Mistake 1: Ignoring Baseline Metrics Before Tuning
3. Mistake 2: Over-Tuning and “Set It and Forget It” Mentality
4. Mistake 3: Misconfiguring Swap Space and Swappiness
5. Mistake 4: Neglecting I/O Performance Bottlenecks
6. Mistake 5: Improper CPU Scaling and Scheduling
7. Mistake 6: Ignoring Memory Leaks and Mismanagement
8. Mistake 7: Mismanaging Network Settings
9. Mistake 8: Relying on Outdated or Inappropriate Tools
10. Mistake 9: Disregarding Application-Specific Tuning
11. Conclusion
12. References

Mistake 1: Ignoring Baseline Metrics Before Tuning

What’s the Mistake?

Jumping straight into tuning without first establishing a performance baseline is like navigating a ship without a map. A baseline is a snapshot of your system’s “normal” behavior—CPU usage, memory consumption, I/O patterns, and network throughput—under typical workloads. Without this, you can’t distinguish between actual bottlenecks and normal fluctuations, leading to unnecessary or even harmful changes.

Why It’s a Problem

• Blind Tuning: You might “fix” a metric that was never broken (e.g., lowering CPU usage on a system where 70% utilization is normal for peak hours).
• Unverifiable Improvements: After making changes, you can’t prove they helped (or hurt) because you lack pre-tuning data.

How to Avoid It

Step 1: Collect Baseline Data
Use tools like sar, vmstat, and iostat to capture metrics over a representative period (e.g., 1–2 weeks, including peak and off-peak hours).

Example commands:

# Collect CPU, memory, I/O, and network stats every 5 seconds for 1 week (1008 samples/day)  
sar -o baseline.sar 5 100800  

# Generate a report from the saved data  
sar -f baseline.sar -u  # CPU usage  
sar -f baseline.sar -r  # Memory usage  
sar -f baseline.sar -b  # I/O transfer rates  

Step 2: Define Key Metrics
Focus on:

• CPU: %user, %system, %iowait (high iowait indicates I/O bottlenecks).
• Memory: free, buffers, cached, swap used.
• I/O: tps (transactions per second), kB_read/s, kB_wrtn/s, await (average I/O latency).
• Network: rxkB/s, txkB/s, %ifutil (interface utilization).

Step 3: Establish “Normal” Ranges
Analyze the baseline to define thresholds (e.g., “normal CPU usage is 40–60% during peak hours”). Use this to flag anomalies later.

Mistake 2: Over-Tuning and “Set It and Forget It” Mentality

What’s the Mistake?

Over-tuning involves tweaking dozens of kernel parameters, application settings, or hardware configurations at once without testing. Even worse is the “set it and forget it” mindset—assuming a one-time tuning will work forever, regardless of changing workloads (e.g., seasonal traffic spikes, new applications).

Why It’s a Problem

• Unintended Consequences: Changing multiple parameters makes it impossible to isolate which tweak caused improvements or failures. For example, increasing net.ipv4.tcp_max_syn_backlog might fix connection issues, but if you also lower vm.dirty_ratio, you could trigger disk I/O storms.
• Stale Tuning: Workloads evolve. A tuning that worked for 100 users may crash with 10,000 users.

How to Avoid It

Step 1: Tune Incrementally
Change one parameter at a time, test its impact against your baseline, and document results. If a tweak doesn’t help, revert it.

Example workflow:

1. Change vm.swappiness from 60 to 10.
2. Monitor for 24 hours using sar and iostat.
3. If swap usage drops and I/O improves, keep it; otherwise, revert to 60.

Step 2: Automate Testing with A/B Comparisons
Use tools like sysbench or perf to simulate workloads before/after tuning. For example:

# Test CPU performance before tuning  
sysbench cpu --cpu-max-prime=20000 run  

# Tune CPU governor to "performance"  
cpupower frequency-set --governor performance  

# Retest and compare results  
sysbench cpu --cpu-max-prime=20000 run  

Step 3: Schedule Regular Re-Tuning
Review and re-test tuning quarterly. Use monitoring tools (e.g., Prometheus + Grafana) to track long-term trends and adjust settings as workloads change.

Mistake 3: Misconfiguring Swap Space and Swappiness

What’s the Mistake?

Swap space is often misunderstood. Common errors include:

• Setting vm.swappiness to 0 (disabling swap entirely) to “improve performance.”
• Allocating too little swap (e.g., 1GB for a 64GB RAM server), leading to Out-of-Memory (OOM) kills.
• Allocating too much swap (e.g., 128GB for 8GB RAM), wasting disk space and slowing I/O.

Why It’s a Problem

• Swappiness = 0: Linux will avoid swap until RAM is completely full, causing sudden OOM kills when memory is exhausted.
• Insufficient Swap: The kernel can’t page out inactive memory, leading to OOM events even if some data could be safely swapped.
• Excessive Swap: Unnecessary I/O as the kernel swaps data that could stay in RAM, increasing latency.

How to Avoid It

Step 1: Understand vm.swappiness
vm.swappiness (0–100) controls how aggressively the kernel swaps memory. Lower values mean the kernel prefers to keep data in RAM; higher values favor swapping inactive pages.

• Servers: Use 10–30 (swap only when RAM is nearly full).
• Desktops/Laptops: Use 60 (balance between responsiveness and battery life).

Set it temporarily with:

sysctl vm.swappiness=10  

Persist it in /etc/sysctl.conf:

echo "vm.swappiness=10" >> /etc/sysctl.conf  
sysctl -p  

Step 2: Allocate Optimal Swap Size
A general rule:

• <2GB RAM: Swap = 2x RAM.
• 2–8GB RAM: Swap = RAM.
• 8–64GB RAM: Swap = 4–8GB (or equal to RAM for hibernation).
• >64GB RAM: Swap = 4–16GB (unless hibernation is needed).

Verify with:

free -h  # Check swap usage  
swapon --show  # List swap devices  

Step 3: Use Swap Files for Flexibility
Instead of fixed swap partitions, use swap files to adjust size dynamically:

fallocate -l 8G /swapfile  
chmod 600 /swapfile  
mkswap /swapfile  
swapon /swapfile  
echo "/swapfile none swap sw 0 0" >> /etc/fstab  

Mistake 4: Neglecting I/O Performance Bottlenecks

What’s the Mistake?

Many admins fixate on CPU and memory while ignoring disk I/O, which is often the root cause of slowdowns. Symptoms include high iowait (CPU waiting for I/O), slow application startups, or database query timeouts.

Common I/O Mistakes

• Using the wrong filesystem (e.g., ext4 for high-throughput databases instead of XFS).
• Not aligning partitions (wastes SSD/NVMe performance).
• Disabling caching (e.g., noatime is good, but nodiratime can hurt directory listing speed).
• Ignoring SSD wear (no TRIM support, leading to performance degradation).

How to Avoid It

Step 1: Identify I/O Bottlenecks
Use tools like iostat, iotop, and dstat to pinpoint issues:

iostat -x 5  # Check per-device I/O stats every 5 seconds  
iotop -o  # Show only processes actively doing I/O  

Key metrics:

• iowait: % of CPU idle waiting for I/O (values >20% indicate a bottleneck).
• await: Average time (ms) for I/O requests (high await = slow disks or misconfigured apps).
• %util: % of time the device is busy (values >80% = saturated I/O).

Step 2: Optimize Filesystems

• High Throughput (e.g., databases): Use XFS (better at handling large files and parallel I/O).
• Stability/Compatibility: Use ext4 (mature, good for general use).
• SSD/NVMe: Enable TRIM to reclaim unused blocks:

  fstrim -av  # Manual TRIM  
  echo "/dev/nvme0n1p2 / ext4 defaults,noatime,discard 0 1" >> /etc/fstab  # Auto-TRIM on mount  

Step 3: Use Caching and Tiering

• LVM Cache: Cache frequent I/O on an SSD using lvcreate --type cache.
• Application-Level Cache: Use Redis or Memcached to offload database reads.
• Kernel Caching: Tune vm.dirty_ratio (max % of RAM for dirty pages before sync) and vm.dirty_background_ratio (background sync threshold). For write-heavy workloads, lower dirty_ratio to 10–20% to reduce I/O bursts.

Mistake 5: Improper CPU Scaling and Scheduling

What’s the Mistake?

CPU performance depends on two factors: scaling (how fast the CPU runs) and scheduling (how processes share CPU time). Mistakes here include:

• Using the “powersave” CPU governor on servers (slows CPUs to save energy, killing performance).
• Not setting process priorities (e.g., letting a backup job starve a critical database).
• Ignoring CPU affinity (binding processes to specific cores, causing cache misses).

Why It’s a Problem

• Underutilized CPUs: A server with 8 cores running at 800MHz (powersave) will perform worse than 4 cores at 3GHz (performance).
• Starvation: Low-priority processes (e.g., nice -19) can hog CPU if not constrained, delaying critical tasks (e.g., nice -20 for Nginx).

How to Avoid It

Step 1: Choose the Right CPU Governor
Use cpupower to set governors:

# List available governors  
cpupower frequency-info --governors  

# Set to "performance" (max speed) for servers  
cpupower frequency-set --governor performance  

# Persist across reboots (systemd example)  
echo "GOVERNOR=performance" > /etc/default/cpufrequtils  
systemctl restart cpufrequtils  

Step 2: Prioritize Processes with nice and ionice

• nice: Adjust CPU priority (-20 = highest, 19 = lowest).

  nice -n -5 /usr/bin/mysqld  # Start MySQL with high CPU priority  

• ionice: Adjust I/O priority (class 1 = realtime, class 2 = best-effort, class 3 = idle).

  ionice -c 3 -p $(pgrep backup.sh)  # Make backup use idle I/O priority  

Step 3: Use CPU Affinity and Cgroups

• Affinity: Bind processes to cores to reduce cache latency (e.g., bind a database to cores 0–3):

  taskset -c 0-3 /usr/bin/mysqld  

• Cgroups: Limit CPU usage for non-critical processes (e.g., limit a container to 2 cores):

  # Create a cgroup  
  mkdir /sys/fs/cgroup/cpu/myapp  
  echo "200000" > /sys/fs/cgroup/cpu/myapp/cpu.cfs_quota_us  # 2 cores (1 core = 100000 us)  
  echo "1000000" > /sys/fs/cgroup/cpu/myapp/cpu.cfs_period_us  
  # Assign PID to cgroup  
  echo <PID> > /sys/fs/cgroup/cpu/myapp/cgroup.procs  

Mistake 6: Ignoring Memory Leaks and Mismanagement

What’s the Mistake?

Assuming “more RAM = better performance” and ignoring memory leaks. Leaks occur when applications allocate memory but fail to release it, leading to gradual RAM exhaustion, OOM kills, or thrashing (excessive swapping).

Why It’s a Problem

• False Sense of Security: Adding RAM masks leaks but doesn’t fix them. A leaky app will eventually exhaust even 256GB of RAM.
• OOM Kills: The kernel’s Out-of-Memory killer (OOM) will terminate processes to free RAM, often killing critical services (e.g., MySQL) instead of the leaky app.

How to Avoid It

Step 1: Monitor Memory Usage
Use vmstat, top, and free to track trends:

vmstat 5  # Check "si" (swap in) and "so" (swap out) for thrashing  
top -o %MEM  # Sort processes by memory usage  

Key metrics:

• RSS (Resident Set Size): Memory the process is actively using (not swapped).
• Swap: si/so: Non-zero values indicate thrashing.

Step 2: Detect Leaks

• Short-Term: Use pmap to check a process’s memory map for growing allocations:

  pmap -x <PID>  # Look for large, increasing anonymous mappings  

• Long-Term: Use valgrind (for development) or systemtap (for production) to trace leaks:

  valgrind --leak-check=full /path/to/app  # Development only (slow!)  

Step 3: Mitigate Leaks

• Restart leaky processes periodically (e.g., use systemd timers to restart a buggy service nightly).
• Use cgroups to limit memory for leaky apps (e.g., memory.limit_in_bytes=4G).
• Fix the root cause: Patch the app or switch to a non-leaky alternative.

Mistake 7: Mismanaging Network Settings

What’s the Mistake?

Network tuning is often overlooked, but misconfigured TCP/IP parameters, DNS, or firewall rules can cripple performance. Common mistakes:

• Using default TCP buffer sizes (too small for high-latency networks like WANs).
• Disabling TCP timestamps (hurts congestion control).
• Not limiting SYN flood attacks (leaving net.ipv4.tcp_max_syn_backlog too low).

Why It’s a Problem

• Slow Transfers: Small TCP buffers cause frequent “stalls” on high-bandwidth links.
• Connection Drops: Low tcp_max_syn_backlog leads to “connection refused” errors under heavy load.
• High Latency: Misconfigured DNS (e.g., no caching) adds 100ms+ delays per request.

How to Avoid It

Step 1: Tune TCP Buffers
Increase net.ipv4.tcp_rmem (read buffer) and net.ipv4.tcp_wmem (write buffer) for WANs:

sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"  # min, default, max (16MB)  
sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"  
sysctl -w net.core.rmem_max=16777216  
sysctl -w net.core.wmem_max=16777216  

Step 2: Enable TCP Timestamps and Congestion Control

• Timestamps: Improve retransmission accuracy (enabled by default in most kernels):

  sysctl -w net.ipv4.tcp_timestamps=1  

• BBR Congestion Control: Better for high-bandwidth, high-latency links (requires kernel ≥4.9):

  sysctl -w net.ipv4.tcp_congestion_control=bbr  

Step 3: Optimize DNS and Firewalls

• DNS Caching: Use dnsmasq or systemd-resolved to cache DNS queries:

  echo "nameserver 127.0.0.1" > /etc/resolv.conf  # Use local cache  

• Firewall Rules: Avoid overly broad iptables rules (e.g., iptables -A INPUT -j ACCEPT). Use ipset for large IP lists to reduce CPU usage.

Mistake 8: Relying on Outdated or Inappropriate Tools

What’s the Mistake?

Using legacy tools like top (limited metrics) or iftop (no per-process network stats) instead of modern alternatives. Even worse is using tools for the wrong job (e.g., free to debug memory leaks instead of pmap).

Why It’s a Problem

• Blind Spots: top shows CPU and memory but not I/O or network per process.
• Inaccuracy: vmstat doesn’t distinguish between buffers and cached memory (critical for understanding free RAM).

How to Avoid It

Modern Toolkit Essentials:

• System Monitoring: htop (interactive, supports mouse, shows CPU cores individually).
• I/O: iotop (per-process I/O), iostat -x (detailed device stats).
• Network: nethogs (per-process bandwidth), ss (replaces netstat, faster and more detailed).
• Memory: smem (shows proportional memory usage), pmap (memory maps).
• Tracing: perf (CPU profiling), bpftrace (eBPF-based tracing for advanced debugging).

Example: Use nethogs to find bandwidth hogs:

nethogs eth0  # Shows which process is using the most network  

Mistake 9: Disregarding Application-Specific Tuning

What’s the Mistake?

Tuning the OS but ignoring application settings. For example, optimizing Linux for high I/O won’t help if your database’s innodb_buffer_pool_size is set to 1GB (too small for 64GB RAM).

Why It’s a Problem

Applications often have their own knobs that dwarf OS tuning. A misconfigured Nginx (worker_processes=1 on an 8-core server) will underperform regardless of OS tweaks.

How to Avoid It

Key Application Tuning Examples:

• Nginx:

  • worker_processes auto (use all CPU cores).
  • worker_connections 10240 (increase for high traffic).
  • tcp_nopush on and tcp_nodelay on (improve TCP performance).

• MySQL/MariaDB:

  • innodb_buffer_pool_size=70% of RAM (caches table data/indexes).
  • innodb_log_file_size=1G (reduce I/O for write-heavy workloads).

• Apache:

  • Use mpm_event instead of mpm_prefork for better concurrency.
  • MaxRequestWorkers=1000 (match to available RAM/CPU).

Use Application-Specific Tools:

• mysqltuner (MySQL/MariaDB):

  wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl  
  perl mysqltuner.pl  # Recommends buffer sizes, connections, etc.  

• apachetop (Apache):

  apachetop -f /var/log/apache2/access.log  # Real-time request stats  

Conclusion

Linux performance tuning is a balancing act—requiring patience, data, and a holistic view of system behavior. By avoiding these common mistakes—starting with baseline metrics, tuning incrementally, and focusing on both OS and application settings—you can unlock your system’s full potential. Remember: measure first, tune second, and never stop monitoring.

References

• Linux Kernel Documentation: sysctl
• Red Hat Enterprise Linux Performance Tuning Guide
• Brendan Gregg’s Linux Performance (book and tools)
• MySQLTuner
• Linux Performance Tools (perf tutorial)