funwithlinux guide

How Linux Handles System Resources: CPU and Memory Management

Linux, the backbone of countless servers, desktops, embedded systems, and cloud infrastructure, is renowned for its efficiency, scalability, and robustness. At the heart of its success lies its sophisticated approach to managing critical system resources: the Central Processing Unit (CPU) and memory. Whether you’re running a high-traffic web server, a personal laptop, or a real-time industrial control system, Linux’s ability to allocate, schedule, and optimize CPU and memory usage directly impacts performance, stability, and responsiveness. In this blog, we’ll dive deep into how Linux manages these two foundational resources. We’ll explore the kernel’s mechanisms for scheduling CPU time among processes, ensuring fairness and efficiency, and how it optimizes memory usage through virtualization, allocation, and recovery. By the end, you’ll have a clear understanding of the "behind-the-scenes" work that makes Linux a top choice for diverse computing environments.

Table of Contents

  1. Understanding System Resources in Linux
  2. CPU Management in Linux
  3. Memory Management in Linux
  4. Tools for Monitoring CPU and Memory Usage
  5. Conclusion
  6. References

Understanding System Resources in Linux

Before delving into management mechanisms, let’s define the resources in question:

  • CPU (Central Processing Unit): The “brain” of the system, responsible for executing instructions from processes and threads. Linux must schedule these tasks efficiently to maximize throughput, minimize latency, and ensure fairness.
  • Memory (Random Access Memory, RAM): Temporary storage for data and instructions that the CPU needs to access quickly. Linux manages memory to ensure processes have the space they need, while avoiding waste and preventing crashes due to exhaustion.

Linux’s resource management is designed to handle diverse workloads—from lightweight embedded devices to multi-socket servers with terabytes of RAM. Its flexibility stems from kernel-level algorithms and subsystems that dynamically adapt to changing conditions.

CPU Management in Linux

The CPU is a finite resource, and with modern systems running hundreds of processes simultaneously, the kernel’s scheduler must decide which process runs next and for how long. This section breaks down Linux’s CPU management strategies.

Process Scheduling: The Basics

At any given time, a Linux system runs dozens of processes (e.g., your browser, terminal, background services). Each process competes for CPU time, but the CPU can only execute one instruction per core at a time. The scheduler (a kernel component) resolves this by:

  • Selecting which process to run next.
  • Allocating a time slice (a short interval) for the process to execute.
  • Preempting (interrupting) the process when its time slice ends or a higher-priority task arrives.

Processes are prioritized using nice values (ranging from -20 to 19, with lower values indicating higher priority) and scheduling classes (e.g., normal, real-time).

The Completely Fair Scheduler (CFS)

Since Linux 2.6.23 (2007), the Completely Fair Scheduler (CFS) has been the default scheduler for normal (non-real-time) processes. Its core goal is to ensure fairness: each process gets a proportional share of CPU time based on its priority.

How CFS Works:

  • Virtual Runtime (vruntime): CFS tracks how much CPU time each process has consumed, adjusted for priority. Higher-priority processes (lower nice values) accumulate vruntime more slowly, ensuring they get more CPU time.
  • Red-Black Tree: Processes are stored in a red-black tree (a self-balancing binary search tree) sorted by vruntime. The scheduler always picks the process with the smallest vruntime (i.e., the one that has used the least CPU time relative to its priority).
  • Time Slice Calculation: CFS avoids fixed time slices. Instead, it calculates a “target latency” (default: 6ms) and divides it among active processes. For example, if 3 processes are running, each gets ~2ms of CPU time before being preempted.

CFS eliminates the “starvation” problem (where low-priority processes are never scheduled) by ensuring even low-priority tasks eventually get CPU time.

CPU Affinity and Isolation

Modern systems have multiple CPU cores. By default, the scheduler moves processes between cores to balance load, but this can cause cache misses (a process’s data is no longer in the core’s cache when it moves).

  • CPU Affinity: Linux allows “pinning” a process to specific cores using taskset or sched_setaffinity(). This reduces cache misses and is useful for performance-critical applications (e.g., databases, real-time systems).
    Example: taskset -c 0,1 my_app pins my_app to cores 0 and 1.

  • CPU Isolation: For real-time workloads, entire cores can be isolated from the general scheduler using the kernel parameter isolcpus=2,3 (isolates cores 2 and 3). Only processes explicitly assigned to these cores (via affinity) will run on them.

Load Balancing Across Cores

While affinity improves cache efficiency, unbalanced load across cores (e.g., one core at 100% usage, others idle) wastes resources. CFS includes a load balancer that:

  • Periodically checks for imbalanced cores (e.g., a core with more runnable processes than others).
  • Moves processes from overloaded cores to underloaded ones, using heuristics to minimize cache disruption (e.g., preferring to move processes with high cache miss rates).

Real-Time Scheduling

For time-critical applications (e.g., industrial control, audio processing), Linux offers real-time scheduling classes that prioritize predictability over fairness:

  • SCHED_FIFO (First-In-First-Out): High-priority processes run until they voluntarily yield the CPU or are preempted by a higher-priority SCHED_FIFO process. No time slicing.
  • SCHED_RR (Round-Robin): Similar to SCHED_FIFO, but processes with the same priority share CPU time via fixed time slices (prevents one process from hogging the CPU).

Real-time processes have priorities from 1 to 99 (higher than normal processes, which use 0). They bypass CFS and are scheduled before all non-real-time tasks.

Memory Management in Linux

Memory management is equally critical: the kernel must allocate RAM to processes, handle limited resources, and abstract physical memory to simplify programming.

Physical vs. Virtual Memory

  • Physical Memory: The actual RAM chips in the system, addressed via physical addresses (e.g., 0x1000 to 0xFFFF).
  • Virtual Memory: An abstraction layer that gives each process a contiguous, private address space (e.g., 0x00000000 to 0xFFFFFFFF on 32-bit systems). The kernel maps virtual addresses to physical addresses using page tables.

Virtual memory enables:

  • Isolation: Processes cannot access each other’s memory (prevents crashes and security breaches).
  • Overcommitment: The kernel can “promise” more virtual memory than available physical RAM (processes rarely use all their allocated memory).
  • Swapping: Inactive memory pages can be moved to disk (swap space) to free RAM for active processes.

Paging and Swapping

  • Paging: Physical and virtual memory are divided into fixed-size pages (typically 4KB). The kernel uses a page table (per process) to map virtual pages to physical pages. If a process accesses a virtual page not in physical memory (a page fault), the kernel loads the page from disk (swap or the executable file) into RAM.

  • Swapping: When physical RAM is low, the kernel moves inactive pages (not recently accessed) to swap space (a disk partition or file). Swapping frees RAM but is slow (disk I/O is ~100,000x slower than RAM). Linux uses a Least Recently Used (LRU) algorithm to select pages for swapping (prioritizing pages not accessed recently).

Note: Swapping is a last resort. Excessive swapping (“thrashing”) occurs when the system spends more time swapping than executing, crippling performance.

Memory Allocation Mechanisms

The kernel must efficiently allocate memory for processes, kernel data structures, and I/O operations. Two key mechanisms handle this:

Buddy System

The buddy system manages physical memory pages for large allocations (e.g., 4KB, 8KB, 16KB, up to the size of a memory zone). It works by:

  • Splitting: Physical memory is divided into blocks of sizes that are powers of two (e.g., 2^0 = 1 page, 2^1 = 2 pages, etc.). To allocate a block of size 3 pages, the system splits a 4-page block into two 2-page buddies, then splits one 2-page block into two 1-page buddies, allocating 2+1 pages.
  • Coalescing: When a block is freed, the buddy system checks if its “buddy” (a block of the same size adjacent in memory) is also free. If so, they merge into a larger block, reducing fragmentation.

The buddy system minimizes external fragmentation (gaps between allocated blocks) but can suffer from internal fragmentation (wasting space in a block larger than needed).

Slab Allocator

For small, frequently allocated objects (e.g., inodes, file descriptors, process descriptors), the slab allocator is used. It:

  • Caches Objects: Creates “slabs” (caches) of objects of the same size (e.g., a cache for 128-byte inode objects).
  • Reuses Freed Objects: When an object is freed, it’s not returned to the buddy system. Instead, it’s kept in the slab cache and reused for future allocations, reducing overhead.
  • Reduces Fragmentation: By reusing objects, the slab allocator avoids the fragmentation caused by frequent small allocations/freeings.

Slab allocators are critical for kernel performance, as the kernel itself allocates millions of small objects (e.g., every open file requires an inode and file descriptor).

The Out-of-Memory (OOM) Killer

When the kernel cannot allocate memory (even after swapping), it invokes the OOM Killer to free memory by terminating one or more processes.

  • OOM Score Calculation: The OOM Killer assigns each process an oom_score (0 to 1000) based on:
    • Memory usage (higher for memory-heavy processes).
    • Nice value (lower-priority processes have higher scores).
    • Whether the process is a “child” (e.g., a browser tab vs. the browser itself).
  • oom_adj: Admins can adjust a process’s score using echo -17 > /proc/<pid>/oom_adj (lower values make the process less likely to be killed; -17 disables OOM killing for the process).

The OOM Killer logs its actions to dmesg (e.g., “Out of memory: Killed process 1234 (chrome)”).

NUMA Support

Multi-socket servers use Non-Uniform Memory Access (NUMA), where each CPU socket has its own “local” memory (faster to access) and “remote” memory (slower, accessed via the system bus). Linux’s NUMA support optimizes memory allocation:

  • Local Allocation: By default, processes are allocated memory from the local NUMA node (the node containing the CPU running the process).
  • NUMA Policies: Admins can override this with policies like interleave (spread allocations across nodes) or preferred (allocate from a specific node) using numactl.

Example: numactl --interleave=all my_app spreads my_app’s memory across all NUMA nodes.

Tools for Monitoring CPU and Memory Usage

To diagnose resource issues, Linux provides powerful monitoring tools:

  • top/htop: Real-time process monitoring. top shows CPU usage (%us, %sy), load average, and memory usage (RES, VIRT). htop adds a user-friendly interface and mouse support.
  • vmstat: Reports system-wide statistics (processes, memory, swap, IO, CPU). Example: vmstat 2 prints stats every 2 seconds.
  • free: Displays memory usage, including buffers/cache (Linux uses free RAM for caching files to speed up access). Example: free -h (human-readable units).
  • sar: Collects historical data (CPU, memory, disk I/O). Install with sysstat; use sar -u 1 5 to check CPU usage 5 times, 1 second apart.
  • numastat: Shows NUMA memory allocation (e.g., local vs. remote memory usage).

Conclusion

Linux’s CPU and memory management mechanisms are a testament to its design philosophy of efficiency, fairness, and scalability. From the CFS scheduler ensuring fair CPU time to the buddy and slab systems optimizing memory allocation, the kernel dynamically adapts to diverse workloads. Understanding these mechanisms empowers system administrators and developers to troubleshoot performance issues, optimize applications, and configure systems for maximum reliability.

Whether you’re running a personal laptop or a cloud server, Linux’s resource management ensures your system runs smoothly—even under heavy load.

References

  1. Linux Kernel Documentation: CPU Scheduler, Memory Management
  2. Bovet, D.P., & Cesati, M. (2015). Understanding the Linux Kernel (3rd ed.). O’Reilly Media.
  3. man pages: sched, taskset, numactl, oom_killer, free, top
  4. Red Hat Documentation: Managing System Memory
  5. Ubuntu Wiki: CPU Scheduling