Once you've determined why you need an intrusion prevention system (IPS), what security, application and protocol coverage, form factor, performance and management needs you have, you'll need to test any IPS you're considering. A test using your own network and traffic is the only test that will tell you whether or not the product is going to meet your requirements.
Although an IPS test doesn't require that you refine your policy completely, you should have a good idea of your network topology and security policy. Without this information, you won't be able to tell whether the IPS can work with your policy.
At this stage, it's also important to look at the flexibility of the IPS configuration. Can you actually express the policy you know you'll want to use in this product? For example, some IPS products don't let you easily manage exception lists for traffic that should not be inspected, or traffic that should bypass specific signatures. If you have a large and diverse network, this kind of flexibility may be important.
The recommended strategy for IPS evaluation is to put the device or combined firewall into alert-only mode. (An IPS that doesn't have alert-only mode should be rejected out of hand.) Rather than actually preventing intrusions, the IPS simply tells you what it would have done. When using this strategy, make sure you let the IPS run for several weeks. Until you build up a set of events, you won't know whether the product can handle the load you're going to offer it.
Once you have some confidence that the IPS isn't going to melt down your network, your evaluation should proceed to full blocking mode. When you do this, make sure you plan sufficient time each day--typically a half day, or more if your network is large or has many Internet-accessible severs--to investigate every alert, and to hunt down the false positives. Even if you haven't taken the time to create a full security policy as part of your evaluation, you should be investigating most alerts. It's critical to get a feel for whether or not the IPS will actually work in your own network.
In any IPS, you should see occasional false positives. These are a natural result of a system that is not tuned. (An IPS that does not throw any false positives ever is probably not actually working.) You should be able to fine-tune the security policy before you go into blocking mode, but still there may be false positives once you go into blocking mode. Be prepared for these, and be prepared to react quickly as they pop up. Also, remember that while clear problems will show up at your help desk in a few seconds, occasional failures may take a week or more before they begin to percolate up into support channels. When planning your testing methodology, allow for sufficient time so these "low and slow" problems will surface.
If you plan to investigate alerts, you should be sure to test the ability of the IPS to support your own "alert lifecycle." Most security managers have a specific methodology they follow to go from alert to qualification to investigation to resolution and finally to policy change. The IPS management system should support your planned methodology and style so that it is easy to handle alerts. You don't want to invest in an IPS that is difficult for you to use.
With blocking enabled, it is also useful to try and stress test the IPS. If you don't have commercial testing tools to inject additional load across the IPS, you can use open source tools that will increase the load of both attack and benign traffic. You may not be able to take the device to its breaking point or to precisely measure the change in behavior, but you should try to increase load by 50 percent or even 100 percent to observe the behavior of the system.
Finally, even though you may be far down an expensive evaluation cycle, it's important to step back and ask yourself whether the product you're considering and the associated capital and operational expenses give you sufficient return-on-investment for the level of security you'll be picking up. While the continuing cost of an IPS is not as high as an IDS would be, the investment in an IPS will range from simply checking a box on a firewall to enable the IPS up to installing devices and management consoles at critical points in your network. Many security professionals go down this path with an idealized idea of the value or effectiveness of IPS products. While IPS can offer significant value in improving the security posture of networks, putting that value into words just before you dive into deployment can help cement the requirements and value for IPS, as well as provide a realistic set of expectations within your organization.
Joel Snyder is a senior partner at Opus One, an IT consulting firm specializing in security and messaging.