nf_conntrack: table full, dropping packet on Nessus server

Issue caused by having iptables rule/s that track connection state. If the number of connections being tracked exceeds the default nf_conntrack table size [65536] then any additional connections will be dropped. Most likely to occur on machines used for NAT and scanning/discovery tools (such as Nessus and Nmap).

Symptoms: Once the connection table is full any additional connection attempts will be blackholed.

 

This issue can be detected using:

$dmesg
nf_conntrack: table full, dropping packet.
nf_conntrack: table full, dropping packet.
nf_conntrack: table full, dropping packet.
nf_conntrack: table full, dropping packet.
...

Current conntrack settings can be displayed using:

$sysctl -a | grep conntrack
net.netfilter.nf_conntrack_generic_timeout = 600
net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 120
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 60
net.netfilter.nf_conntrack_tcp_timeout_established = 432000
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_last_ack = 30
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close = 10
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300
net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300
net.netfilter.nf_conntrack_tcp_loose = 1
net.netfilter.nf_conntrack_tcp_be_liberal = 0
net.netfilter.nf_conntrack_tcp_max_retrans = 3
net.netfilter.nf_conntrack_udp_timeout = 30
net.netfilter.nf_conntrack_udp_timeout_stream = 180
net.netfilter.nf_conntrack_icmpv6_timeout = 30
net.netfilter.nf_conntrack_icmp_timeout = 30
net.netfilter.nf_conntrack_acct = 0
net.netfilter.nf_conntrack_events = 1
net.netfilter.nf_conntrack_events_retry_timeout = 15
net.netfilter.nf_conntrack_max = 65536
net.netfilter.nf_conntrack_count = 1
net.netfilter.nf_conntrack_buckets = 16384
net.netfilter.nf_conntrack_checksum = 1
net.netfilter.nf_conntrack_log_invalid = 0
net.netfilter.nf_conntrack_expect_max = 256
net.ipv6.nf_conntrack_frag6_timeout = 60
net.ipv6.nf_conntrack_frag6_low_thresh = 196608
net.ipv6.nf_conntrack_frag6_high_thresh = 262144
net.nf_conntrack_max = 65536

To check the current number of connections being tracked by conntrack:

/sbin/sysctl net.netfilter.nf_conntrack_count

Options for fixing the issue are:

1. Stop using stateful connection rules in iptables (probably not an option in most cases)
2. Increase the size of the connection tracking table (also requires increasing the conntrack hash table)
3. Decreasing timeout values, reducing how long connection attempts are stored (this is particularly relevant for Nessus scanning machines that can be configured to attempt many simultaneous port scans across an IP range)

 

Making the changes in a persistent fashion RHEL 6 examples:

# 2: Increase number of connections
echo "net.netfilter.nf_conntrack_max = 786432" >> /etc/sysctl.conf
echo "net.netfilter.nf_conntrack_buckets = 196608" >> /etc/sysctl.conf
# Increase number of bucket to change ration from 1:8 to 1:4 (more # memory use but better performance)
echo 'echo "196608" > /sys/module/nf_conntrack/parameters/hashsize' >> /etc/rc.local

# 3: Alter timeout values
# Generic timeout from 10 mins to 1 min
echo "net.netfilter.nf_conntrack_generic_timeout = 60" > /etc/sysctl.conf

# Change unacknowledged timeout to 30 seconds (from 10 mins)
echo "net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 30" > /etc/sysctl.conf

# Change established connection timeout to 1 hour (from 10 days)
echo "net.netfilter.nf_conntrack_tcp_timeout_established = 3600" > /etc/sysctl.conf

These changes will persist on reboot.

To apply changes without reboot run the following:

sysctl -p
echo "196608" > /sys/module/nf_conntrack/parameters/hashsize

To review changes:

sysctl -a | grep conntrack

Reference and further reading: http://antmeetspenguin.blogspot.com.au/2011/01/high-performance-linux-router.html