funwithlinux guide

Managing Linux Services with Systemd: A Beginner's Guide

If you’ve ever used a Linux system, you’ve likely interacted with "services"—background processes that power everything from web servers (like Nginx) to network managers (like `systemd-networkd`). Whether you’re running a home server, a cloud instance, or just your daily Linux desktop, understanding how to manage these services is a critical skill. In modern Linux distributions, **systemd** has emerged as the de facto "init system"—the software responsible for booting the system, managing services, and coordinating system processes. Replacing older systems like SysVinit and Upstart, systemd offers powerful features like parallelized service startup, on-demand activation, and integrated logging. This guide is designed for beginners to demystify systemd and teach you how to control, monitor, and troubleshoot Linux services with confidence. By the end, you’ll be comfortable using `systemctl` (systemd’s main command-line tool), interpreting service statuses, viewing logs, and even creating your own custom services.

Table of Contents

What is Systemd?

Systemd is a system and service manager for Linux operating systems. It is the first process started during boot (with PID 1) and acts as the “parent” of all other processes. Its primary roles include:

  • Booting the system by starting essential services and daemons.
  • Managing running services (starting, stopping, restarting).
  • Handling service dependencies (e.g., ensuring a web server starts only after the network is up).
  • Logging system events via the journald daemon.

Since its adoption around 2015, systemd has become the standard init system for most major Linux distributions, including Ubuntu, Fedora, Debian, CentOS, RHEL, and Arch Linux. Its ubiquity makes learning systemd a must for anyone working with Linux.

Understanding Systemd Units

At the heart of systemd are units—configuration files that define system resources and their relationships. Units come in several types, but the most common (and relevant to service management) is the service unit (.service files). Other unit types include sockets (.socket), targets (.target, similar to SysVinit runlevels), and timers (.timer, for scheduling tasks).

Service Unit Files

Service units describe how to manage a service (e.g., start/stop commands, dependencies, user permissions). They are stored in standard directories:

  • /lib/systemd/system/: Default units provided by the OS or installed packages (e.g., Nginx, Apache).
  • /etc/systemd/system/: Custom or overridden units (user-defined or modified services).
  • ~/.config/systemd/user/: User-specific units (run in user space, not system-wide).

Anatomy of a .service File

A typical .service file has three main sections: [Unit], [Service], and [Install]. Let’s break down each with an example (a hypothetical myapp.service):

[Unit]
Description=My Custom Application
After=network.target  # Start only after the network is up
Requires=mysql.service  # Depend on MySQL; if MySQL fails, this service fails too

[Service]
User=ubuntu  # Run the service as the "ubuntu" user
Group=ubuntu  # Run with the "ubuntu" group
ExecStart=/usr/local/bin/myapp  # Command to start the service
Restart=on-failure  # Restart if the service fails (e.g., crashes)
WorkingDirectory=/opt/myapp  # Set the working directory
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"  # Set environment variables

[Install]
WantedBy=multi-user.target  # Start when the system reaches the multi-user runlevel
  • [Unit]: Metadata and dependencies (e.g., Description, After, Requires).
  • [Service]: How the service runs (e.g., ExecStart command, user/group, restart policy).
  • [Install]: How the service is installed (e.g., WantedBy defines which target triggers the service on boot).

Key Systemd Commands: An Overview

The primary tool for interacting with systemd is systemctl (system control). It lets you start/stop services, check statuses, enable/disable auto-start, and more. Think of systemctl as the “remote control” for systemd.

Here’s a quick preview of essential systemctl commands we’ll cover:

  • systemctl start <service>: Start a service.
  • systemctl stop <service>: Stop a service.
  • systemctl restart <service>: Restart a service.
  • systemctl enable <service>: Start the service automatically on boot.
  • systemctl disable <service>: Prevent the service from starting on boot.
  • systemctl status <service>: Check if a service is running.

Managing Services

Let’s dive into the most common service management tasks with systemctl.

Starting, Stopping, and Restarting Services

Start a Service

To start a service immediately (without waiting for a reboot), use:

sudo systemctl start <service-name>

Example: Start the Nginx web server:

sudo systemctl start nginx

Stop a Service

To halt a running service:

sudo systemctl stop <service-name>

Example: Stop the Apache web server:

sudo systemctl stop apache2

Restart a Service

If a service’s configuration changes (e.g., you edit /etc/nginx/nginx.conf), restart it to apply changes:

sudo systemctl restart <service-name>

Example: Restart Nginx after editing its config:

sudo systemctl restart nginx

Reload a Service (Graceful Restart)

For services that support it (e.g., Nginx, Apache), reload applies config changes without interrupting active connections:

sudo systemctl reload <service-name>

Example: Reload Nginx gracefully:

sudo systemctl reload nginx

Enabling and Disabling Services on Boot

Services often need to start automatically when the system boots (e.g., a web server should start after a reboot). Use enable/disable to control this behavior.

Enable a Service (Auto-Start on Boot)

To configure a service to start on boot:

sudo systemctl enable <service-name>

Example: Ensure Nginx starts on boot:

sudo systemctl enable nginx

Systemd creates symlinks in /etc/systemd/system/ (e.g., multi-user.target.wants/nginx.service) to trigger the service at boot.

Disable a Service (Prevent Auto-Start)

To stop a service from starting on boot (but leave it manually startable):

sudo systemctl disable <service-name>

Example: Disable Apache from auto-starting:

sudo systemctl disable apache2

Mask and Unmask (Advanced)

For services you never want to start (even manually), use mask (it “locks” the service):

sudo systemctl mask <service-name>  # Prevent start
sudo systemctl unmask <service-name>  # Allow start again

Checking Service Status

To verify if a service is running, use systemctl status <service-name>. This command provides a wealth of information, including:

  • Whether the service is active (running), inactive, or failed.
  • The process ID (PID) and memory usage.
  • A snippet of recent logs.

Example: Check Nginx Status

sudo systemctl status nginx

Sample output:

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2024-03-20 12:34:56 UTC; 2h 10min ago
       Docs: man:nginx(8)
   Main PID: 1234 (nginx)
      Tasks: 2 (limit: 1152)
     Memory: 3.5M
     CGroup: /system.slice/nginx.service
             ├─1234 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             └─1235 nginx: worker process

Key details:

  • Loaded: The service file path and whether it’s enabled on boot.
  • Active: active (running) means it’s working; inactive means stopped; failed means it crashed.
  • Main PID: The process ID of the service’s main process.

Quick Status Checks

For scripting or quick verification, use these shortcuts:

  • systemctl is-active <service>: Returns active, inactive, or failed.
    Example: systemctl is-active nginx → active.
  • systemctl is-enabled <service>: Returns enabled, disabled, or masked.
    Example: systemctl is-enabled nginx → enabled.

Viewing Logs with Journalctl

Systemd uses journald (a logging daemon) to collect and store logs from services, the kernel, and other system components. The journalctl command lets you query these logs, making it easy to debug misbehaving services.

Basic Journalctl Commands

View Logs for a Specific Service

To see logs for a service (e.g., Nginx), use -u (unit):

journalctl -u nginx

Follow Live Logs

To “tail” a service’s logs in real time (like tail -f), add -f:

journalctl -u nginx -f

Filter by Time

Use --since and --until to narrow logs to a time range:

journalctl -u nginx --since "2024-03-20 12:00" --until "2024-03-20 14:00"

Or relative times: --since "1 hour ago", --until "30 minutes ago".

Show Recent Logs

Limit to the last N entries with -n:

journalctl -u nginx -n 20  # Show last 20 logs

Persistent vs. Volatile Journals

By default, some distributions store logs in memory (volatile), meaning they’re lost after a reboot. To enable persistent logging (logs survive reboots), run:

sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

Creating Custom Service Files

Creating a custom service lets you run your own scripts or applications as background services (e.g., a Python bot, a Node.js API, or a backup script). Let’s walk through an example.

Step 1: Create a Simple Script

First, create a script to run as a service. For this example, we’ll use a bash script that logs messages to a file:

sudo nano /usr/local/bin/logger-script.sh

Add this content:

#!/bin/bash
while true; do
    echo "[$(date)] Service is running..." >> /var/log/custom-service.log
    sleep 60  # Log every 60 seconds
done

Make it executable:

sudo chmod +x /usr/local/bin/logger-script.sh

Step 2: Create the Service File

Next, create a .service file in /etc/systemd/system/ (system-wide services):

sudo nano /etc/systemd/system/custom-logger.service

Add this configuration:

[Unit]
Description=Custom Logging Service
After=network.target

[Service]
User=root
ExecStart=/usr/local/bin/logger-script.sh
Restart=always  # Restart if the script exits (even normally)
WorkingDirectory=/tmp

[Install]
WantedBy=multi-user.target

Step 3: Reload Systemd and Start the Service

Systemd doesn’t automatically detect new service files. Reload its configuration:

sudo systemctl daemon-reload

Now start and enable the service:

sudo systemctl start custom-logger
sudo systemctl enable custom-logger

Verify It Works

Check the service status:

sudo systemctl status custom-logger

View the logs (from both the service and the script):

journalctl -u custom-logger  # Systemd logs for the service
cat /var/log/custom-service.log  # Script-generated logs

Troubleshooting Common Service Issues

Even experienced users run into service problems. Here’s how to diagnose and fix common issues.

Issue: Service Fails to Start

Check the status for clues:

sudo systemctl status <service>

Look for “Failed” in the output and read the error message (e.g., “Executable path not found”).

Issue: “Unit .service not found”

This means systemd can’t find your service file.

  • Ensure the service file exists in /etc/systemd/system/ or /lib/systemd/system/.
  • Run sudo systemctl daemon-reload after creating/editing the file.

Issue: Service Starts but Immediately Stops

Check the Restart policy in the .service file. If set to on-failure, the service won’t restart if it exits normally. Use Restart=always for scripts that run in loops (like our logger example).

Issue: Logs Show Permission Errors

If the service can’t read/write files, ensure the User/Group in the .service file has the right permissions. For example, if the script writes to /var/log/, use User=root (or add the user to the adm group).

Validate Service Files

Use systemd-analyze to check for syntax errors in service files:

sudo systemd-analyze verify custom-logger.service

Conclusion

Systemd is a powerful tool for managing Linux services, and with systemctl and journalctl, you now have the skills to control, monitor, and troubleshoot services like a pro. From starting a web server to creating custom background services, these tools will serve as your foundation for Linux system administration.

Remember, practice makes perfect! Experiment with common services (e.g., nginx, ssh, docker) to get comfortable with systemctl commands. And when in doubt, check the status and logs—they’re your best friends for debugging.

References