Secure Linux SSH Connections: Best Practices

Table of Contents

1. Understanding SSH and Its Vulnerabilities
2. Key-Based Authentication: Replace Passwords
3. Hardening the SSH Daemon (sshd_config)
4. Network-Level Security for SSH
5. Monitoring and Auditing SSH Activity
6. Advanced: SSH Certificates and Jump Hosts
7. Conclusion
8. References

1. Understanding SSH and Its Vulnerabilities

What is SSH?

SSH (Secure Shell) is a cryptographic network protocol that replaces insecure tools like Telnet and FTP. It uses public-key cryptography to authenticate users and encrypt data in transit, ensuring confidentiality and integrity. The SSH ecosystem includes:

• ssh: Client tool to initiate connections.
• sshd: Server daemon running on the target machine.
• ssh-keygen: Generates SSH key pairs.
• ssh-copy-id: Copies public keys to remote servers.

Common SSH Vulnerabilities

Even with encryption, SSH is not invulnerable. Key risks include:

• Weak Passwords: Brute-force attacks (automated tools trying thousands of password combinations).
• Default Configurations: Unchanged default ports (22), permissive access controls, or verbose logging disabled.
• Unpatched sshd: Exploits targeting outdated SSH daemon versions (e.g., CVE-2024-6387 for OpenSSH).
• Insider Threats: Overly permissive user/group access or unmonitored activity.
• Man-in-the-Middle (MitM) Attacks: Rare with modern SSH (thanks to host key verification), but possible if clients ignore warnings.

2. Key-Based Authentication: Replace Passwords

Passwords are inherently weak: users reuse them, choose simple phrases, or store them insecurely. SSH key-based authentication eliminates password risks by using cryptographic key pairs (public/private) for authentication.

How SSH Keys Work

• Private Key: Stored securely on your local machine (never shared).
• Public Key: Uploaded to the remote server’s ~/.ssh/authorized_keys file.
• When connecting, the server challenges the client to prove ownership of the private key matching the public key, ensuring only authorized users gain access.

Step 1: Generate an SSH Key Pair

On your local machine, run:

ssh-keygen -t ed25519 -C "[email protected]"  

• -t ed25519: Uses the Ed25519 algorithm (more secure and faster than RSA).
• -C: Adds a comment (e.g., email) to identify the key.

When prompted, set a strong passphrase for the private key (adds an extra layer if the key is stolen).

Step 2: Copy the Public Key to the Server

Use ssh-copy-id to securely transfer the public key:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@remote_server_ip  

If ssh-copy-id isn’t available, manually copy the public key:

cat ~/.ssh/id_ed25519.pub | ssh user@remote_server_ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"  

Step 3: Verify Key Authentication

Test the connection (you should now log in without a password, but may need to enter the private key passphrase):

ssh user@remote_server_ip  

Step 4: Disable Password Authentication

Once key-based auth works, disable password logins in sshd_config (see Section 3 for details).

3. Hardening the SSH Daemon (sshd_config)

The SSH daemon (sshd) is configured via /etc/ssh/sshd_config. Tweaking this file is critical for security. Always back up the config first:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak  

Critical sshd_config Directives

Example Hardened sshd_config

# Basic Security  
Port 2222  
PermitRootLogin no  
PubkeyAuthentication yes  
PasswordAuthentication no  
ChallengeResponseAuthentication no  
PermitEmptyPasswords no  

# Access Control  
AllowUsers [email protected]/24 [email protected]  
MaxAuthTries 3  
LoginGraceTime 30s  

# Session Security  
X11Forwarding no  
TCPKeepAlive yes  
ClientAliveInterval 300  # Send keepalive every 5 minutes  
ClientAliveCountMax 3    # Disconnect after 3 failed keepalives  

# Logging & Performance  
UseDNS no  
LogLevel VERBOSE  
Banner /etc/ssh/banner.txt  # Optional: Legal warning for unauthorized access  

Apply Changes and Test

After editing, validate the config and restart sshd:

sudo sshd -t  # Checks for syntax errors  
sudo systemctl restart sshd  

Critical: Open a new terminal and test SSH access before closing your current session (avoids locking yourself out).

4. Network-Level Security for SSH

Even with a hardened sshd, network-level controls add another defense layer.

Firewall Rules: Restrict SSH Access

Use a firewall to limit SSH port access (e.g., only allow trusted IPs).

UFW (Uncomplicated Firewall, Ubuntu/Debian):

# Allow SSH on custom port (e.g., 2222) from specific IP  
sudo ufw allow from 192.168.1.0/24 to any port 2222 proto tcp  
sudo ufw enable  

Iptables (Advanced):

# Allow SSH from 10.0.0.0/24 on port 2222  
sudo iptables -A INPUT -p tcp --dport 2222 -s 10.0.0.0/24 -j ACCEPT  
# Block all other SSH traffic  
sudo iptables -A INPUT -p tcp --dport 2222 -j DROP  

Block Brute-Force Attacks with Fail2ban

Fail2ban monitors logs for repeated failed SSH attempts and temporarily bans malicious IPs.

Install Fail2ban:

sudo apt install fail2ban  # Ubuntu/Debian  
sudo dnf install fail2ban  # RHEL/CentOS  

Configure Fail2ban

Create a custom config file (overrides defaults):

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local  

Edit /etc/fail2ban/jail.local to harden SSH rules:

[sshd]  
enabled = true  
port = 2222  # Match your SSH port  
filter = sshd  
logpath = /var/log/auth.log  
maxretry = 3  # Ban after 3 failed attempts  
bantime = 3600  # Ban for 1 hour (3600 seconds)  
findtime = 600  # Look for attempts in the last 10 minutes (600s)  
ignoreip = 192.168.1.0/24  # Ignore trusted IP ranges  

Restart Fail2ban:

sudo systemctl restart fail2ban  
sudo systemctl enable fail2ban  # Start on boot  

Use a VPN for SSH Access

Restrict SSH to only be accessible via a VPN (e.g., WireGuard, OpenVPN). This ensures attackers can’t even reach the SSH port without first authenticating to the VPN.

5. Monitoring and Auditing SSH Activity

Detecting attacks early requires monitoring SSH logs and setting up alerts.

SSH Logs Location

SSH logs are stored in /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS). Use grep to filter activity:

# Failed login attempts  
sudo grep "Failed password" /var/log/auth.log  

# Successful logins  
sudo grep "Accepted publickey" /var/log/auth.log  

Centralized Logging

For multiple servers, aggregate logs with tools like:

• ELK Stack: Elasticsearch, Logstash, Kibana (visualize SSH activity).
• Graylog: Open-source log management with alerting.

Alerting on Suspicious Activity

Use scripts or tools to trigger alerts for anomalies (e.g., 10+ failed attempts in 5 minutes).

Example bash script to monitor failed logins and send email alerts:

#!/bin/bash  
LOG_FILE="/var/log/auth.log"  
THRESHOLD=5  
RECIPIENT="[email protected]"  

FAILED_ATTEMPTS=$(grep -c "Failed password" $LOG_FILE)  

if [ $FAILED_ATTEMPTS -gt $THRESHOLD ]; then  
  echo "ALERT: $FAILED_ATTEMPTS failed SSH attempts detected on $(hostname)" | mail -s "SSH Brute-Force Alert" $RECIPIENT  
fi  

Add this to crontab to run hourly:

0 * * * * /path/to/script.sh  

6. Advanced: SSH Certificates and Jump Hosts

For enterprise environments, these advanced techniques simplify management and enhance security.

SSH Certificates (Instead of Keys)

SSH keys work well for small teams, but managing hundreds of keys is cumbersome. SSH certificates use a Certificate Authority (CA) to sign keys, enabling centralized revocation and expiration.

Step 1: Generate a CA Key

On a secure machine (not the server), generate a CA key:

ssh-keygen -t ed25519 -f ca_key -C "SSH CA"  

Step 2: Sign User Keys

Sign a user’s public key with the CA:

ssh-keygen -s ca_key -I "alice" -n "alice" -V +365d ~/.ssh/id_ed25519.pub  

• -I: Certificate identity.
• -n: Allowed username(s).
• -V: Expiration (1 year).

Step 3: Configure the Server to Trust the CA

Add the CA public key to /etc/ssh/sshd_config:

TrustedUserCAKeys /etc/ssh/ca_key.pub  

Jump Hosts (Bastion Servers)

A jump host acts as a single entry point for SSH access to internal servers, reducing exposure.

Configure ~/.ssh/config on your local machine:

Host jump_host  
  HostName jump.example.com  
  User alice  
  IdentityFile ~/.ssh/id_ed25519  

Host internal_server  
  HostName 10.0.0.100  
  User alice  
  ProxyJump jump_host  # Routes SSH through jump_host  

Now connect directly to the internal server via the jump host:

ssh internal_server  

7. Conclusion

Securing SSH requires a layered approach: strong authentication (keys/certificates), hardened daemon configs, network controls (firewalls, Fail2ban), and vigilant monitoring. By following these best practices, you’ll mitigate 99% of common SSH attacks, from brute-force attempts to misconfiguration exploits.

Final Checklist:
✅ Use Ed25519 keys with passphrases.
✅ Disable password authentication and root login.
✅ Restrict SSH via firewall and AllowUsers.
✅ Enable Fail2ban and verbose logging.
✅ Regularly audit logs and update sshd.

8. References

• OpenSSH Official Documentation
• SSH Academy: Key Management
• Fail2ban Configuration Guide
• NIST Special Publication 800-113 (SSH Security)
• Ubuntu Server Hardening Guide