Once you've decided which type of intrusion prevention system (signature-, rate-, or behavior-based) offers you...
the best capabilities for your requirements, a second decision comes into play: what application and protocol coverage do you want?
For signature-based IPS, the quality (much more than the quantity) of the signatures and the underlying detection engine determines whether a product meets your needs. Even in signature-less systems, such as rate-based or behavioral IPS, coverage of applications and protocols will vary from product to product.
Some IPS products have evolved from intrusion detection engines, and are sometimes glibly referred to as "IDS with the IPS bit turned on". In evaluating IPS products, you may run into ones with literally thousands of signatures, a sign of an IDS that has been repurposed as an IPS. An IPS's job is not to detect every possible attack or reconnaissance attempt; an IPS only has the job of preventing intrusions. That job takes considerable smarts. The signature count of a well-designed IPS engine will number in the hundreds, not thousands. This doesn't mean that an IPS cannot have additional signatures and also serve as an IDS, as long as the IPS function is primary and the IDS function is secondary.
However, an IDS which has simply been put into inline mode with its thousands of signatures will offer a lower degree of security against serious attackers than an IPS designed from the ground up as a prevention, rather than as a detection, system. For example, because the IDS signatures are designed to detect attacks, even attacks that might never succeed, they will have a higher false positive rate than IPS signatures that are designed to identify and block working attacks. At the same time, the burden of having thousands of signatures to inspect traffic across many ports and in many directions will cause performance issues (such as high latency or lost packets) that might be acceptable in an IDS, but would never be allowed in an IPS.
Merely taking an IDS device from a monitoring port and moving it directly into an in-line IPS position without significantly re-engineering the system is a recipe for failure.
Neither the IDS detection engine nor the IDS signature set is very appropriate for an IPS deployment for two reasons: First, there are fundamental differences in design philosophy between IDS detection engines and IPS detection engines; Secondly, simply repurposing IDS signatures into an IPS will create a fairly inaccurate IPS. For example, an IDS should behave differently when it alerts depending on the susceptibility of a destination system. That is, an IDS might handle a Code Red attack on a Unix server (which is not vulnerable) differently from a Code Red attack on a Windows IIS server (which is vulnerable). However, the IPS doesn't need to make any of those judgment calls. It simply has to block Code Red attacks altogether, a significantly easier job.
On the other hand, an IPS doesn't always have the simpler job. Fundamentally, the penalty for a false positive in an IDS scenario is low, which means that there is an incentive to err on the side of producing false positives. Conversely, the penalty for a false positive detection in an IPS deployment is huge, which means that the IPS designer must have a significantly different mindset.
For example, when the team at Oulu University in Finland discovered widespread security vulnerabilities in common SNMP implementations, IDS vendors created signatures that quickly matched the exploits used by the Oulu PROTOS project. However, detecting an exploit is not the same as detecting vulnerability. Our testing has shown that these SNMP signatures both have a high false positive rate in that they detect valid uses of the protocol and mark them as exploit attempts, as well as a high false negative rate in that they fail to detect very simple and slight variations that a moderately skilled attacker might use to evade an IDS.
In this situation, an IPS has the more difficult job because it must not block valid SNMP calls, yet it should detect attempts to exploit these vulnerabilities. Here, the IPS must combine the use of protocol analysis with signature detection. Protocol analysis is also invaluable in providing zero-day coverage to help catch exploits against previously undiscovered vulnerabilities.
Finally, when considering security and coverage, you should look at the potential IPS actions. A simple IPS can only drop offending packets, but more sophisticated actions are also available with more advanced products. These actions include resetting connections (in one or both directions), sending alerts, capturing packets, blocking future traffic, and even changing configuration of other network devices, such as firewalls. While the more advanced actions, such as blocking future traffic (sometimes called "shunning" after Cisco's terminology) or changing firewall access control lists, may seem attractive, our experience is that these cause more problems than they solve. Thus, making these capabilities a requirement in picking your IPS could be overkill, and could force you to disregard perfectly capable devices.
Joel Snyder is a senior partner at Opus One, an IT consulting firm specializing in security and messaging.