Tip

Windows rootkit detection tools and tactics

    Requires Free Membership to View

Send Tom your security questions

Have a security question about risk management? Windows security? Mobile devices? Send them to Tom, and he'll answer them in a future tip.

A SearchMidmarketSecurity.com reader asks our expert Tom Chmielarski, "I'm concerned that my Windows machine has a rootkit - what is the best way to detect and remove it?"

A typical virus or worm does not exactly advertise its presence, but its components are there to find if you know what to look for.

A rootkit, on the other hand, is an item of malware that is particularly stealthy; it hooks itself into the operating system and will not be shown by typical system utilities. It is often used to mask the presence of other malware on the system. A rootkit will usually be configured to hide additional malware, too, such as a keylogger or botnet-node. Keep in mind: rootkits exist on every major operating system platform and are certainly not unique to Windows.

I'm going to answer this question backwards and address removal first. Since the inherent nature of the rootkit is to hide itself, detection and removal of a rootkit is a tricky process. You can't simply use Regedit, the Microsoft Registry Editor, to find the registry keys that the malware is using; the rootkit will hide them.

When dealing with any malware that has had the opportunity to execute itself, modify your system, and then download other malware from the Internet, the best response is to back up your data and reinstall the operating system and applications from known-good media. While this scorched-earth type response may seem a bit extreme, it is the only way to really ensure you've removed it. The malware may have added users, changed system files, modified your OS's kernel, and performed any number of other unwanted changes. You may remove part of the compromise, but it is difficult to ensure that you've really corrected the damage. Even if you have removed the malicious component, the system changes themselves may linger and result in instability or unexpected system behavior.

More "Ask the Expert" responses

A reader asks Tom Chmielarski, "What do you consider to be the trade-offs of going with a best-of-breed product over all-in-one security software like Forefront?"

 Now let's move on to detecting a rootkit. Many antivirus products will detect rootkits with various levels of success. Generally speaking, you shouldn't expect the "scan now" antivirus to detect a rootkit. The AV product, however, may identify a dropper, or installer program, that wants to set up that rootkit. If the rootkit gets installed through a vulnerability -- perhaps using an unpatched Web browser flaw that is exploited when you visit a website that serves malicious advertisements -- then the antivirus tool probably won't help.

Since the primary objective of the rootkit is to hide itself, a good way to find a rootkit is to enumerate your system's contents and boot up using a known-good operating system. A bootable CD is an excellent choice for this and I've previously discussed creating a Bootable Windows CD. The default UBCD comes with RootKitty, an application for detecting rootkits by comparing the contents found when booted normally and when booted via UBCD. Microsoft also offers a free tool, RootkitRevealer, to do this type of search.

Another means to detect a rootkit, or other malware, is the use of a packet sniffer, such as WinDump, or a network firewall to detect the traffic emanating from your computer. If you start the suspect computer and, without direct user interaction, that computer tries to connect to IP addresses outside of your company (particularly universities and home Internet services), then the chances are good that it has something unwanted.

Windows rootkit detection and prevention is a complicated topic, and I'm only skimming the surface here. As a final thought on this topic I'd like to remind you of the infamous Sony rootkit from 2005. This rootkit was installed as a DRM component on computers that autoplayed certain Sony CDs. (Note: Mark Russinovich, author of the Microsoft RootkitRevealer, discovered the rootkit program). The removal of this corporate-designed software was tricky and potentially harmful to the computer, and this saga may be interesting for anyone who would like to know more about rootkits.

Tom Chmielarski is a senior consultant with GlassHouse Technologies, Inc.

Send Tom your security questions.

Join us on LinkedIn.

This was first published in June 2010

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.