High-Availability Firewalls with iptables and HAProxy

Table of Contents

1. Understanding High Availability in Firewalls
2. iptables Overview: The Linux Firewall Workhorse
3. HAProxy Overview: Load Balancing and Failover Management
4. Why Combine iptables and HAProxy for HA Firewalls?
5. Prerequisites
6. Step 1: Setting Up iptables on HA Nodes
7. Step 2: Configuring HAProxy for Load Balancing and Failover
8. Step 3: Setting Up Keepalived for Virtual IP (VIP) Management
9. Testing the High-Availability Firewall Setup
10. Monitoring the HA Firewall Cluster
11. Best Practices for Production
12. Troubleshooting Common Issues
13. Conclusion
14. References

1. Understanding High Availability in Firewalls

High availability (HA) refers to a system’s ability to operate continuously without downtime, even when individual components fail. For firewalls, HA is critical because:

• Security Gaps: A single firewall failure can leave the network unprotected.
• Business Continuity: Downtime disrupts access to services, applications, and data.
• Compliance: Many industries (e.g., finance, healthcare) require 99.99%+ uptime for regulatory compliance.

Key HA components for firewalls include:

• Redundancy: Multiple identical firewall nodes (active-active or active-passive).
• Failover: Automatic traffic redirection to healthy nodes when a failure is detected.
• Load Balancing: Distributing traffic across nodes to prevent overload.

2. iptables Overview

iptables is a Linux kernel-based firewall utility that filters network traffic based on predefined rules. It operates at the packet level, enforcing security policies like:

• Allowing/blocking traffic by IP, port, protocol, or application.
• Network Address Translation (NAT) for internal-to-external communication.
• Logging suspicious traffic for auditing.

Key Concepts:

• Tables: Predefined rule sets (e.g., filter for packet filtering, nat for NAT).
• Chains: Rule sequences within tables (e.g., INPUT for incoming traffic, OUTPUT for outgoing traffic).
• Rules: Conditions (e.g., --dport 22 for SSH) and actions (e.g., ACCEPT, DROP, LOG).

Limitation:

iptables alone cannot provide HA—its rules are static to a single node, and it lacks built-in load balancing or failover mechanisms.

3. HAProxy Overview

HAProxy (High Availability Proxy) is a open-source load balancer and proxy server. It excels at:

• Traffic Distribution: Routing requests across multiple backend servers (e.g., web servers, firewalls).
• Health Checks: Detecting failed backend nodes and diverting traffic to healthy ones.
• SSL Termination: Decrypting/encrypting traffic to reduce backend server load.

Key Features for HA:

• Load Balancing Algorithms: Round-robin, least connections, source IP hashing, etc.
• Active-Passive/Active-Active Modes: Supports both failover (active-passive) and load-sharing (active-active) setups.
• Stats Interface: Real-time monitoring of traffic, backend status, and errors.

Role in Firewalls:

HAProxy acts as a “traffic cop,” ensuring traffic reaches healthy firewall nodes and avoiding failed ones.

4. Why Combine iptables and HAProxy for HA Firewalls?

iptables and HAProxy complement each other perfectly for HA firewalls:

• iptables: Enforces security policies (e.g., blocking malicious IPs, restricting ports) on individual nodes.
• HAProxy: Manages traffic distribution, failover, and health checks across nodes.

Together, they provide:

• Security + Availability: iptables secures the network, while HAProxy ensures uptime.
• Consistency: Traffic is filtered by identical iptables rules across all nodes, avoiding security gaps during failover.
• Scalability: Add more nodes to handle increased traffic without reconfiguring clients.

5. Prerequisites

Before building your HA firewall cluster, ensure you have:

6. Step 1: Setting Up iptables on HA Nodes

Both firewall nodes must have identical iptables rules to ensure consistent security during failover. Follow these steps on both nodes:

6.1 Install iptables and Persistence Tools

# Ubuntu/Debian  
sudo apt update && sudo apt install iptables iptables-persistent -y  

# RHEL/CentOS  
sudo dnf install iptables-services -y  
sudo systemctl enable --now iptables  

6.2 Define Base Rules

Set default policies to DROP (deny all traffic) and allow essential services:

# Flush existing rules  
sudo iptables -F  

# Default policies  
sudo iptables -P INPUT DROP  
sudo iptables -P FORWARD DROP  
sudo iptables -P OUTPUT ACCEPT  

# Allow loopback traffic (critical for internal services)  
sudo iptables -A INPUT -i lo -j ACCEPT  

# Allow SSH (restrict to management IPs for security)  
sudo iptables -A INPUT -p tcp --dport 22 -s 192.168.1.0/24 -j ACCEPT  

# Allow HTTP/HTTPS (adjust ports for your use case)  
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT  
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT  

# Allow ICMP (ping) for health checks  
sudo iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT  

# Allow traffic to VIP (critical for HAProxy frontend)  
sudo iptables -A INPUT -d 192.168.1.200/32 -p tcp --dport 80 -j ACCEPT  
sudo iptables -A INPUT -d 192.168.1.200/32 -p tcp --dport 443 -j ACCEPT  

6.3 Save Rules Persistently

# Ubuntu/Debian  
sudo netfilter-persistent save  

# RHEL/CentOS  
sudo service iptables save  

Verify: Ensure rules are identical on both nodes with sudo iptables -L -v.

7. Step 2: Configuring HAProxy for Load Balancing and Failover

HAProxy will route traffic to the active firewall node(s) and detect failures via health checks.

7.1 Install HAProxy

# Ubuntu/Debian  
sudo apt install haproxy -y  

# RHEL/CentOS  
sudo dnf install haproxy -y  

7.2 Configure HAProxy

Edit /etc/haproxy/haproxy.cfg on both nodes (use the VIP for the frontend):

global  
    log /dev/log local0  
    maxconn 2000  
    user haproxy  
    group haproxy  
    daemon  

defaults  
    log global  
    mode tcp  # Use "http" for application-level filtering  
    option tcplog  
    option dontlognull  
    retries 3  
    timeout connect 5s  
    timeout client 50s  
    timeout server 50s  

# Frontend: Listen on VIP and route to backends  
frontend firewall_frontend  
    bind 192.168.1.200:80  # VIP and port  
    bind 192.168.1.200:443  
    default_backend firewall_nodes  

# Backend: Define firewall nodes with health checks  
backend firewall_nodes  
    mode tcp  
    balance roundrobin  # Distribute traffic evenly  
    option tcp-check  # TCP-level health check  
    tcp-check connect port 80  # Check if port 80 is responsive  
    server fw-node1 192.168.1.10:80 check fall 3 rise 2  # Node 1  
    server fw-node2 192.168.1.11:80 check fall 3 rise 2  # Node 2  

# Stats page (access via http://VIP:9000/stats)  
listen stats  
    bind *:9000  
    stats enable  
    stats uri /stats  
    stats auth admin:SecurePass123!  # Secure with credentials  

7.3 Validate and Restart HAProxy

# Check config syntax  
sudo haproxy -c -f /etc/haproxy/haproxy.cfg  

# Restart service  
sudo systemctl restart haproxy  
sudo systemctl enable haproxy  

8. Step 3: Setting Up Keepalived for Virtual IP (VIP) Management

HAProxy uses a VIP (192.168.1.200) for client traffic. Keepalived (based on VRRP) manages the VIP, ensuring it floats to a healthy node during failures.

8.1 Install Keepalived

# Ubuntu/Debian  
sudo apt install keepalived -y  

# RHEL/CentOS  
sudo dnf install keepalived -y  

8.2 Configure Keepalived on Master Node

Edit /etc/keepalived/keepalived.conf on fw-node1 (master):

vrrp_instance VI_1 {  
    state MASTER  
    interface eth0  # Replace with your network interface  
    virtual_router_id 51  # Must be identical on all nodes  
    priority 100  # Higher than backup (e.g., 90)  
    advert_int 1  # VRRP heartbeat interval (1s)  

    # Authentication  
    authentication {  
        auth_type PASS  
        auth_pass SecureVRRPpass!  # Same on all nodes  
    }  

    # Virtual IP (VIP)  
    virtual_ipaddress {  
        192.168.1.200/24  
    }  

    # Track HAProxy status: Demote if HAProxy fails  
    track_script {  
        check_haproxy  
    }  
}  

# Script to check HAProxy health  
vrrp_script check_haproxy {  
    script "pidof haproxy"  # Check if haproxy process is running  
    interval 2  # Check every 2s  
    weight -20  # Lower priority by 20 if failed  
}  

8.3 Configure Keepalived on Backup Node

On fw-node2 (backup), use the same config but set state BACKUP and priority 90.

8.4 Start Keepalived

sudo systemctl restart keepalived  
sudo systemctl enable keepalived  

9. Testing the High-Availability Setup

Validate failover, traffic distribution, and security with these tests:

9.1 Verify VIP Assignment

# On master node (should show VIP)  
ip addr show eth0 | grep 192.168.1.200  

# Output should include: inet 192.168.1.200/24 scope global eth0  

9.2 Test Failover

1. Stop HAProxy on the master:

   sudo systemctl stop haproxy  

2. Check the VIP on the backup node—it should now be active:

   ip addr show eth0 | grep 192.168.1.200  

3. Restart HAProxy on the master—the VIP will migrate back once health checks pass.

9.3 Validate Traffic Flow

Use curl or a browser to access http://192.168.1.200. Traffic should route to either node. Check HAProxy stats (http://192.168.1.200:9000/stats) to confirm load balancing.

10. Monitoring the HA Firewall Cluster

Proactive monitoring ensures you catch issues before they cause downtime:

• HAProxy Stats: Use the /stats page to track backend health, request rates, and errors.
• Keepalived Logs: Monitor /var/log/syslog for VRRP failover events.
• Iptables Logs: Log denied traffic with iptables -A INPUT -j LOG --log-prefix "DENIED: " and monitor /var/log/kern.log.
• Prometheus + Grafana: Use haproxy_exporter and node_exporter for metrics visualization.

11. Best Practices for Production

• Sync Configs: Use Ansible, Puppet, or Git to keep iptables/HAProxy rules identical across nodes.
• Limit Access: Restrict SSH and HAProxy stats to trusted IPs only.
• Encrypt Traffic: Use HAProxy for SSL termination and iptables to block unencrypted ports.
• Test Regularly: Schedule monthly failover tests to validate HA behavior.
• Backup Configs: Store iptables, HAProxy, and Keepalived configs in a secure, version-controlled location.

12. Troubleshooting Common Issues

13. Conclusion

By combining iptables (for security) and HAProxy (for traffic management), paired with Keepalived (for VIP failover), you can build a robust, high-availability firewall cluster. This setup ensures your network remains secure and accessible even when individual nodes fail.

Remember: HA is an ongoing process—regular testing, monitoring, and config updates are critical to maintaining uptime.

14. References

• iptables Documentation
• HAProxy Configuration Guide
• Keepalived VRRP Guide
• Ubuntu iptables Persistence
• VRRP RFC 5798