Because every network requires a different way of measuring performance and most UTM products offer thousands of deployment options, it's hard to draw even general conclusions about how these products will behave in your network. However, we can say that most enterprises will want to proceed cautiously when adding UTM features, such as intrusion-prevention systems and antivirus scanning, to their perimeter firewall boxes, because of their unpredictable impact on total system performance.
In our baseline testing with only the firewalling capabilities turned on, eight of the 14 appliances easily exceeded our 1Gbps measurement goal. When we turned on their UTM features, however, systems that breezed through the 1,000Mbps mark slowed dramatically. Out of 56 test results collected with various UTM features turned on, 36 registered results that were 250Mbps or less. Read the latest WhitePaper - NAC: A Multi-Symptom Remedy
With IPS configuration, your choice of signatures can make the difference between a speedy firewall and a snail. The top IPS performer, IBM Internet Security Systems' Proventia MX5010, shows that you can get a high-speed IPS riding on top of a firewall. Other platforms require careful tuning and an educated selection of what you want to protect before you can achieve predictable and acceptable performance.
Antivirus scanning has a similar cost in most platforms (the Fortinet FortiGate 3600A is an exception) that also makes it a dangerous add-on, taking some platforms to their knees and turning gigabit firewalls into megabit slowpokes.
We ran baseline traffic through the firewalls using Spirent Communications' Avalanche and Reflector load testing products. We set up a load of 1Gbps spread across four ports, with the Reflector serving up Web pages on 20 simulated Web servers on two of the ports, and Avalanche simulating 500 Web clients on the other two.
In each case where we faced a deployment-option question, we optimize for security rather than speed. Yet one man's security might be another man's overkill, especially when the performance penalty for these security features is significant.
Enabling HTTP inspection, a feature that provides some intrusion prevention, caused almost no performance penalty in our Cisco ASA5540, reducing throughput from 660Mbps to about 640Mbps. Enabling HTTP inspection and choosing an advanced feature (such as blocking ActiveX content) caused an 80% drop in total throughput.
Picking a configuration for performance measurements got more complicated when we tested with UTM features enabled. Check Point Software's IPS technology, called Secure Defense, is a good example. With several hundred IPS options for different types of applications and different attacks, there is no way just to turn on IPS. You have to decide which of the signatures you want to use. When you turn on anything above the default settings, the performance impact is huge.
When we tested the Nokia IP290 running Check Point's firewall software with Secure Defense disabled, and then enabled with default settings, we saw a tiny performance hit (from 1003M to 993Mbps). When we followed Check Point's recommended settings for providing IPS for servers (which scans for more attacks), we saw an 85% drop in performance.
To get our IPS performance results, we used two scenarios -- one asking the firewalls to protect servers and one asking them to protect client systems. With server-protective IPS, there are more potential attacks, but the IPS doesn't have to look at as much traffic. For example, in our HTTP testing, it took about 20Mbps of traffic to a server to generate 1000Mbps of traffic coming back from it. Server-protective IPS has to look only at the traffic to the server.
On all firewalls, we set up a modest policy, letting HTTP through between segments with network address translation (NAT) enabled. We weren't trying to find out the top speed for each of the products; most of the boxes we tested had stated capacities faster than our 1Gbps test bed. Our objective was to ascertain how much of a drop we were going to find when we turned on UTM features.
The security features of many of the firewalls we tested comprise a spectrum of options. For example, Secure Computing will let you run the Sidewinder with packet filters or a generic proxy, neither of which have the same security model as the full application-aware proxy it also supports. With packet filters, the Sidewinder maxed out our test bed; with a generic proxy it nearly hits 1Gbps. However, any enterprise paying the US$80,000 price tag would do so for the full proxy capabilities. When we turned those on, raw performance fell to a respectable 826Mbps.