Monitoring ip_conntrack on zabbix
I needed some statistics and triggers on zabbix of ip_conntrack. Here is how to monitor it, of course You can expand it as You wish.
First of all You need to add some configuration to Your zabbix agents. Put those lines at the end of zabbix config (in my case it was: /etc/zabbix/zabbix_agentd.conf):
UserParameter=ip_conntrack_count,cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
UserParameter=ip_conntrack_max,cat /proc/sys/net/ipv4/netfilter/ip_conntrack_maxAnd then restart zabbix agent:
/etc/init.d/zabbix-agent restartNow You can configure zabbix, so let’s login and import this file:
From now on, You are able to monitor the state of ip_conntrack on Your system.
Some readers may be ieernesttd to know what ip_conntrack is in the first place, and why it fills up. If you run an iptables firewall, and have rules that act upon the state of a packet, then the kernel uses ip_conntrack to keep track of what state what connections are in so that the firewall rule logic can be applied against them. If you have a system that’s getting a lot of network activity (high rates of connections, lots of concurrent connections, etc) then the table will accumulate entries. The entries remain until an RST packet is sent from the original IP address. If you have a flaky network somewhere between you, and the clients accessing your server, it can cause the RST packets to be dropped due to the packet loss, and leave orphaned entries in your ip_conntrack table. This can also happen if you have a malfunctioning switch or NIC card not necessarily a routing problem out on the internet somewhere.Typically when I’ve seen this trouble crop up is when a server is the target of a DDoS attack. Filling up the ip_conntrack table is a relatively easy way to knock a server off line, and attackers know this.As Major suggested, you can get short term relief by increasing the size of the table. However, these entries are held in memory by the kernel. The bigger you make the table, the more memory it will consume. That memory could be used by your server to serve requests if you really don’t need the stateful firewall capability. Don’t waste resources on this feature if you really don’t need it. Another option to consider is turning OFF iptables rules that use ip_conntrack so the state able is not used at all. Anything with -m state or -t nat can be turned off. If you want to just flush all your iptables rules you can do an iptables -P to set a default allow policy and iptables -F to flush all the rules. On an RHEL or CentOS system you can just do service iptables stop .Once iptables is no longer using ip_conntrack, you can reclaim the memory the table was using by unloading the related kernel modules.rmmod ipt_MASQUERADErmmod iptable_natrmmod ipt_statermmod ip_conntrackThen you will have an empty ip_conntrack that will stay empty. I mention this because a lot of sysadmins have hordes of iptables rules installed as a matter of course, and don’t recognize the downside of having them present. You can still use iptables, but to avoid the use of ip_conntrack simply don’t use rules based on stateful logic.