funwithlinux guide

Building Secure VPNs with iptables and OpenVPN

In an era where data privacy and network security are paramount, Virtual Private Networks (VPNs) have become a cornerstone of secure communication. Whether you’re protecting remote workforces, securing IoT devices, or anonymizing internet traffic, a well-configured VPN is essential. Among the most popular VPN solutions, **OpenVPN** stands out for its flexibility, open-source nature, and robust encryption. However, even the strongest VPN can be undermined by poor firewall configuration. This is where **iptables**—Linux’s built-in firewall—comes into play. In this guide, we’ll walk through building a secure VPN using OpenVPN and hardening it with iptables. You’ll learn how to set up an OpenVPN server, generate secure certificates, configure client connections, and enforce strict firewall rules to block unauthorized access. By the end, you’ll have a production-ready VPN that balances usability with enterprise-grade security.

Table of Contents

  1. Understanding OpenVPN and iptables

    • 1.1 What is OpenVPN?
    • 1.2 What is iptables?
    • 1.3 Why Combine OpenVPN and iptables?

  2. Prerequisites

  3. Step 1: Setting Up OpenVPN

    • 3.1 Install OpenVPN and Easy-RSA
    • 3.2 Generate Certificates and Keys with Easy-RSA
    • 3.3 Configure the OpenVPN Server
    • 3.4 Configure OpenVPN Clients

  4. Step 2: Securing the VPN with iptables

    • 4.1 Understanding iptables Basics
    • 4.2 Default Policies: Deny All Incoming Traffic
    • 4.3 Allow OpenVPN and Critical Services
    • 4.4 Enable VPN Traffic Forwarding
    • 4.5 Restrict Client Access with iptables Rules
    • 4.6 Save iptables Rules Persistently

  5. Advanced Security Hardening

    • 5.1 Harden OpenVPN Configuration
    • 5.2 iptables Rate Limiting (Brute-Force Protection)
    • 5.3 Enable Two-Factor Authentication (2FA)
    • 5.4 Encrypt VPN Traffic with Strong Ciphers

  6. Testing Your Secure VPN

    • 6.1 Verify VPN Connection
    • 6.2 Validate iptables Rules
    • 6.3 Security Audits with Tools Like nmap

  7. Troubleshooting Common Issues

  8. Conclusion

  9. References

Understanding OpenVPN and iptables

1.1 What is OpenVPN?

OpenVPN is an open-source VPN protocol that uses SSL/TLS for encryption. It operates in two modes:

  • TUN (Tunnel): Routes IP packets between networks (ideal for remote access).
  • TAP (Tap): Simulates an Ethernet bridge (used for layer-2 connectivity, e.g., local network access).

OpenVPN is highly customizable, supporting various authentication methods (certificates, passwords, 2FA) and encryption algorithms (AES, ChaCha20).

1.2 What is iptables?

Iptables is a command-line firewall utility for Linux that filters network traffic based on predefined rules. It operates on chains (INPUT, OUTPUT, FORWARD) and tables (filter, nat, mangle) to control traffic flow. For VPNs, iptables ensures only authorized traffic reaches the OpenVPN server and restricts what VPN clients can access.

1.3 Why Combine OpenVPN and iptables?

OpenVPN secures the VPN tunnel, but iptables secures the server itself and controls VPN client behavior. For example:

  • Block unauthorized access to the OpenVPN port.
  • Prevent VPN clients from accessing sensitive internal services.
  • Limit bandwidth abuse or DDoS attacks on the VPN server.

Prerequisites

Before starting, ensure you have:

  • A Linux server (Ubuntu 20.04/22.04, Debian 11, or CentOS Stream 9 recommended).
  • Root or sudo access to the server.
  • A public IP address (for remote client access).
  • Basic familiarity with Linux command-line and networking.

Install dependencies upfront:

# Ubuntu/Debian
sudo apt update && sudo apt install -y openvpn easy-rsa iptables-persistent

# CentOS/RHEL
sudo dnf install -y openvpn easy-rsa iptables-services

Step 1: Setting Up OpenVPN

1.1 Install OpenVPN and Easy-RSA

We’ll use Easy-RSA to generate the Public Key Infrastructure (PKI) needed for OpenVPN (CA, server/client certificates, and encryption keys).

# Copy Easy-RSA scripts to a working directory
sudo cp -r /usr/share/easy-rsa/{3,EasyRSA}
cd EasyRSA/3

# Initialize PKI
sudo ./easyrsa init-pki

# Build Certificate Authority (CA) – store the CA key securely (offline!)
sudo ./easyrsa build-ca nopass  # Use "nopass" for automation; add a passphrase for production!

When prompted, enter a “Common Name” (e.g., “MyVPN-CA”).

1.2 Generate Server and Client Certificates

Next, generate keys for the OpenVPN server and clients:

Server Certificates:

# Generate server certificate and key
sudo ./easyrsa build-server-full server nopass

# Generate Diffie-Hellman (DH) parameters (for key exchange)
sudo ./easyrsa gen-dh

# Generate TLS-auth key (optional but recommended for HMAC protection)
openvpn --genkey secret keys/tls-auth.key

Client Certificates (Repeat for Each Client):

# Replace "client1" with a unique name for each client
sudo ./easyrsa build-client-full client1 nopass

1.3 Configure the OpenVPN Server

Create a server configuration file (/etc/openvpn/server.conf):

# Basic settings
port 1194                  # Standard UDP port (use TCP 443 if UDP is blocked)
proto udp                  # UDP is faster for VPNs; TCP for reliability
dev tun                    # Use TUN for IP routing

# Certificate paths (update these paths!)
ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/server.crt
key /etc/openvpn/pki/private/server.key  # Keep this file secure!
dh /etc/openvpn/pki/dh.pem
tls-auth /etc/openvpn/pki/tls-auth.key 0  # 0 = server side

# VPN subnet (clients will get IPs in this range)
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt  # Track client IPs

# Push routes to clients (e.g., allow access to server's LAN or internet)
push "route 192.168.1.0 255.255.255.0"  # Optional: Local LAN access
push "dhcp-option DNS 1.1.1.1"          # Cloudflare DNS
push "dhcp-option DNS 8.8.8.8"          # Google DNS

# Security hardening
keepalive 10 120           # Send ping every 10s; restart after 120s of inactivity
cipher AES-256-GCM         # Strong encryption (authenticated cipher)
auth SHA512                 # HMAC hash for data integrity
tls-version-min 1.3         # Require TLS 1.3
tls-cipher TLS-AES-256-GCM-SHA384
tls-auth /etc/openvpn/pki/tls-auth.key 0  # HMAC to prevent tampering
user nobody                 # Run OpenVPN as non-root user
group nogroup
persist-key
persist-tun

# Logging
status openvpn-status.log
log-append openvpn.log
verb 3                      # Adjust verbosity (1=quiet, 9=debug)

Copy Certificates to OpenVPN Directory:

sudo mkdir -p /etc/openvpn/pki
sudo cp -r EasyRSA/3/pki/{ca.crt,issued,private,dh.pem,tls-auth.key} /etc/openvpn/pki/

1.4 Configure OpenVPN Clients

Create a client configuration file (client1.ovpn):

client
dev tun
proto udp
remote YOUR_SERVER_PUBLIC_IP 1194  # Replace with your server's IP

resolv-retry infinite
nobind
user nobody
group nogroup
persist-key
persist-tun

# Certificate paths (embed these or reference local files)
<ca>
-----BEGIN CERTIFICATE-----
# Paste contents of /etc/openvpn/pki/ca.crt here
-----END CERTIFICATE-----
</ca>
<cert>
-----BEGIN CERTIFICATE-----
# Paste contents of /etc/openvpn/pki/issued/client1.crt here
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN PRIVATE KEY-----
# Paste contents of /etc/openvpn/pki/private/client1.key here
-----END PRIVATE KEY-----
</key>
<tls-auth>
-----BEGIN OpenVPN Static key V1-----
# Paste contents of /etc/openvpn/pki/tls-auth.key here
-----END OpenVPN Static key V1-----
</tls-auth>

remote-cert-tls server
tls-version-min 1.3
cipher AES-256-GCM
auth SHA512
verb 3

Distribute client1.ovpn to users (e.g., via secure email).

Step 2: Securing the VPN with iptables

2.1 Understanding iptables Basics

Iptables rules are processed in order. We’ll use the filter table (default) to control traffic and the nat table to handle VPN routing.

2.2 Default Policies: Deny All Incoming Traffic

Start with a strict baseline: block all incoming traffic except explicitly allowed services.

# Flush existing rules
sudo iptables -F
sudo iptables -t nat -F

# Set default policies
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT ACCEPT  # Allow outgoing traffic (adjust if needed)

2.3 Allow Critical Services

Permit essential traffic (SSH, OpenVPN):

# Allow loopback traffic (required for server internal communication)
sudo iptables -A INPUT -i lo -j ACCEPT

# Allow established/related connections (e.g., VPN clients after initial handshake)
sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH (restrict to your IP for security: -s YOUR_IP/32)
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow OpenVPN (UDP 1194)
sudo iptables -A INPUT -p udp --dport 1194 -j ACCEPT

2.4 Enable VPN Traffic Forwarding

To let VPN clients access the internet or local networks, enable IP forwarding and configure iptables:

# Enable IP forwarding (persist across reboots)
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# Allow VPN traffic to be forwarded (TUN interface)
sudo iptables -A FORWARD -i tun0 -j ACCEPT
sudo iptables -A FORWARD -o tun0 -m state --state ESTABLISHED,RELATED -j ACCEPT

# Masquerade VPN client traffic (NAT) to allow internet access
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE  # Replace "eth0" with your server's WAN interface

2.5 Restrict Client Access with iptables Rules

Prevent VPN clients from accessing sensitive services (e.g., internal databases):

# Block clients from accessing 192.168.1.100 (example: a database server)
sudo iptables -A FORWARD -i tun0 -d 192.168.1.100 -j DROP

# Allow clients to access only HTTP/HTTPS on the internet
sudo iptables -A FORWARD -i tun0 -p tcp --dport 80 -j ACCEPT
sudo iptables -A FORWARD -i tun0 -p tcp --dport 443 -j ACCEPT

2.6 Save iptables Rules Persistently

By default, iptables rules reset after reboot. Save them:

# Ubuntu/Debian (using iptables-persistent)
sudo netfilter-persistent save

# CentOS/RHEL
sudo service iptables save

Advanced Security Hardening

3.1 Harden OpenVPN Configuration

Add these to server.conf for extra security:

tls-timeout 120           # Re-negotiate TLS every 2 minutes
crl-verify crl.pem        # Revoke compromised certificates (generate with Easy-RSA)
comp-lzo no               # Disable compression (avoids CRIME attacks)
push "comp-lzo no"        # Enforce no compression on clients

3.2 iptables Rate Limiting (Brute-Force Protection)

Limit SSH/OpenVPN login attempts to block brute-force attacks:

# Limit SSH to 5 attempts per minute
sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH
sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 60 --hitcount 5 --name SSH -j DROP

# Limit OpenVPN to 3 connections per minute per IP
sudo iptables -A INPUT -p udp --dport 1194 -m state --state NEW -m recent --set --name VPN
sudo iptables -A INPUT -p udp --dport 1194 -m state --state NEW -m recent --update --seconds 60 --hitcount 3 --name VPN -j DROP

3.3 Enable Two-Factor Authentication (2FA)

Add 2FA using openvpn-auth-pam (PAM module for Linux):

# Install PAM plugin
sudo apt install -y openvpn-auth-pam

# Update server.conf to use PAM authentication
echo 'plugin /usr/lib/openvpn/openvpn-auth-pam.so login' | sudo tee -a /etc/openvpn/server.conf

# Configure PAM to use Google Authenticator (optional)
sudo apt install -y libpam-google-authenticator

3.4 Encrypt VPN Traffic with Strong Ciphers

Use modern ciphers like AES-256-GCM (authenticated encryption) and TLS 1.3 to avoid vulnerabilities like Heartbleed.

Testing Your Secure VPN

4.1 Verify VPN Connection

On a client, connect with:

openvpn --config client1.ovpn

Check IP address:

curl ifconfig.me  # Should show the server's public IP

4.2 Validate iptables Rules

List active rules:

sudo iptables -L -v  # Verbose list of filter rules
sudo iptables -t nat -L  # List NAT rules

4.3 Security Audits with nmap

Scan the server from an external network to ensure only OpenVPN/SSH ports are open:

nmap YOUR_SERVER_PUBLIC_IP -p 22,1194  # Should show "open" for allowed ports

Troubleshooting Common Issues

  • Connection Refused: Check if openvpn@server is running (sudo systemctl status openvpn@server) and iptables allows port 1194.
  • No Internet Access: Ensure net.ipv4.ip_forward=1 is set and the MASQUERADE rule exists in nat table.
  • Certificate Errors: Verify certificate paths and expiration dates (openssl x509 -in cert.crt -noout -dates).

Conclusion

By combining OpenVPN’s encrypted tunnels with iptables’ granular traffic control, you’ve built a secure VPN that protects both the server and clients. Remember to:

  • Rotate certificates regularly.
  • Update OpenVPN and iptables packages.
  • Audit logs (openvpn.log) for suspicious activity.

Stay secure!

References