Type of rootkit: User mode
Installed by user action, such as clicking phish links or hitting bad Web pages. Often include escalation of privileges to gain deeper access to the kernel.
How to defend against it: Make sure browsers are secure, also deploy up-to-date antivirus/intrusion prevention, endpoint security and network gateway protections.
Type of rootkit: Kernel Mode
Kernel rootkits exist for all major operating systems. In May, proof-of-concept on Cisco IOS was delivered by a Core Security researcher at EuSecWest, London.
How to defend against it: Antivirus has a hard time detecting kernel rootkits because antivirus runs at the application layer and rootkits run with full control of the kernel. To put antimalware at a higher level of privilege than kernel, look into Virtual Machine Manager-based antimalware, recently introduced as VMSafe by VMware.
Type of rootkit: Packages
Rootkits such as Rustock.C spread like kernel-level viruses and launch spam bots. This packaging is creating some confusion as to what constitutes a rootkit and what constitutes a bot (remote controlled computer).
How to defend against it: Tune desktop and network monitoring tools to look for signs of viral, bot and other malware making calls, opening connections and so on. Because these packages can even turn off desktop defenses, gateway monitoring is critical. Watch for anomalous inbound and especially any outbound behavior. Also look for encrypted traffic, which controllers use to run bot commands over IRC.
Type of rootkit: Kernel and Hardware
These "persistent" rootkits run in the kernel and then hide themselves in the microprocessor when the computer turns off. Researcher John Heassman's rootkit hides in firmware's APCI (Advanced Computer and Power Interface) and reloads at BIOS. Gamebot rootkit packages are using this technology.
How to defend against it: At this level, current endpoint security technologies are not useful; and cleaning is difficult because the rootkit reinstalls at pre-boot when the machine powers on. Technologies like Intel's Trusted Platform Module Trusted Boot Process are doing cryptographic signing of loaded boot drivers to and from the kernel. However, it will be years until enough processors are replaced or introduced in new systems to make a difference.
Type of rootkit: Hardware Rootkits
Proof of concept of rootkit for SMM (System Management Mode, which controls basic functions such as sleep and fans) scheduled to be delivered at BlackHat 08.
How to defend against it: Move monitoring and diagnostics down to the processor. There is some market movement in this direction with a recent Microsoft acquisition and network diagnostics looking at this layer.
Type of rootkit: Virtual rootkits
Proof of concepts such as Joanna Rutkowska's BluePill for AMD processors (BlackHat 06) have not been found in the wild and are believed to be more trouble than they're worth because kernel mode rootkits are still quite successful.
How to defend against it: Novell and other virtual machine providers have management tools that can catch rogue machines. So can virtual machine antivirus, such as VMSafe.