Frequently speakers will pronounce the typical virus detection scheme of pattern matching “doesn’t work” and (for dramatic emphasis or hyperbole) is “dead.” The argument is that pattern files miss so many malicious files. See, for example, Eighty percent of new malware defeats anti-virus, wherein
the general manager of the Australian Computer Emergency Response Team (AusCERT), Graham Ingram, told the audience that popular desktop anti-virus applications “don’t work”.
“At the point we see it as a CERT, which is very early on — the most popular brands of anti-virus on the market … have an 80 percent miss rate. That is not a detection rate that is a miss rate.
“So if you are running these pieces of software, eight out of 10 pieces of malicious code are going to get in,” said Ingram.
This should not be a surprising finding. The population of malware that Graham Ingram drew samples from is the population of newly-released malware samples. As Ingram points out, malware developers test their variants against anti-virus software to avoid detection using current pattern files. They have many tools at their disposal, and can obfuscate their threat to evade pattern matching systems. That is, the samples were designed to avoid detection and many (80%) were successful.
The leap from “misses 80 percent of newly released malware samples” to “doesn’t work” or “is dead” does not get clarified.
Analogously, suppose you attended a presentation about lock security. During the presentation, all padlocks were opened without keys or combinations. All models of door locks were picked. If the conclusion you draw is “locks are dead,” then you have exposed how little you understand about the role locks play.
Ryan Naraine (in Anti-Virus Is Dead, D-E-A-D, Dead!) cites Mr. Williams as justification for his claim that:
The spyware guys are having a field day playing — and winning — cat-and-mouse with AV vendors. Quick spam run with a new Trojan; sit back and watch the AV guys scramble to ship signatures; tweak the code, send another spam run, watch and giggle as another round of .DAT files get built; repeat, rinse, dry.
Similarly, Amrit Williams blogged that Anti-virus is Dead.
Stand-alone, signature-based, anti-virus is dead. The stand-alone anti-spyware market is over too, if it even existed!
Signature based AV isn’t protecting anyone anymore, it certainly wasn’t providing any protection against spyware or some of the nastier threats that have popped up recently.
Bottom Line: By the end of 2007 stand-alone AV will be dead, d-e-a-d, dead! Organizations need to evolve their client security programs or expect to see increased costs as the number of agents continues to rise.
Amrit Williams argues that because anti-virus isn’t detecting new variations of threats, its isn’t providing any protection. Mr. Williams also argues that if a system is compromised, it provides no protection from outbound (botnet) traffic and does not detect rootkits that have been installed.
In an August 2009 whitepaper, Cyveillance reports that anti-virus vendors are generally unable to detect that day’s newly released malware.
As the results show, even the most popular AV solutions detect less than half of the latest malware threats. So if you visit a malicious Web site you could have a more than 1 in 2 chance of being infected with malware.
This should not be a surprising finding. What is surprising is that signature-based, preventative methods fared as well as they did against newly-released malware.
A Cisco blog posting (The Effectiveness of Antivirus on New Malware Samples by Kevin Timm December 21, 2009), confirms that newly discovered malware samples escape anti-virus detection. The situation is significantly improved a week later.
A SANS handler diary posting “Is Anti-Virus Dead?” by John Bambenek argues that
anti-virus by its very nature is reactive… it will only block against known threats.
That is its role. Anti-virus software blocks against known threats. The virus pattern updates are reactions to known threats. Once deployed, anti-virus software provides defensive protection against known threats.
Mr. Bambenek also points out that anti-virus solutions never have been sufficient. Exactly.
Chris Brenton has a “Why anti-virus is dead presentation” posted.
In Introducing Stealth Malware Taxonomy, Joanna Rutkowsky provides an image of anti-virus vendors:
The A/V industry has developed lots of mechanisms to determine whether a given executable is “bad” or “good”, such as behavior monitoring, sandboxing, emulation, AI based heuristics and not to mention all the signature based approaches.
In Rutkowska: Anti-Virus Software Is Ineffective (Ryan Narain in e-week October 26, 2006). Ms. Rutkowsky had demonstrated Blue Pill at Black Hat. At the presentation and in the interview, Ms. Rutkowsky explained how undetectable some attacks can be. She, too, seems to have expectations about anti-virus software that would not have been within the product specifications.
I’m not very impressed with existing anti-virus solutions, especially for the Windows platform. They all concentrate on finding “the bad” instead of verifying that system is in a “good” shape.
So, we can see very sophisticated technology employed by anti-virus products to handle various .exe-packers and decide whether the .exe file in question is “good” or “bad.”
The advantage of malware: The PolyPack project illustrates the advantage malware developers have over malware detectors. A malware developer has a variety of tools (“packers”) which will rearrange their code. Packers can be used on packed code. Malware developers find an arrangement that avoids malware detection and release it. Someone must find a sample of the malware variation (loosely called “strain of virus”) and get it to malware detection organizations.
The problem of scope: Over the years, anti-virus vendors have adopted more and more responsibility for detecting various forms of malware. An anti-virus vendor who restricted their scope to viruses (employing the SANS definition of virus, where malicious code attaches to trusted code) would have no customers. The customer expectation is to detect worms, Trojan Horse programs, spyware, at least some of the hacking tools. If enough customers complain that their hackings tools have legitimate uses, the anti-virus vendor will drop detection. Anti-virus vendors face legal challenges when they detect Trojan Horse programs. After all, the user chose to install the program (invited the Trojan Horse in). The vendor of the Trojan Horse program can accuse the anti-virus vendor of restraint of trade, as in Zango v. Kaspersky.
Through this scope creep, customers have come to rely upon anti-virus software to enforce the detection of undesirable software. What is undesirable is not consistent across customers, although there is a significant body which is considered malware. It is a poorly defined list which with general agreement Anti-virus vendors willingly encourage this attitude. Disappointment arises when vendor and customer don’t share expectations.
People sometimes choose to install a keylogger to monitor their child’s, spouse’s or employee’s activity. Should anti-virus vendors detect keyloggers, such as Perfect Keylogger, as malware? Someone needed to choose to install Perfect Keylogger, and it is detected as malware but many anti-virus vendors. Competitors, such as Spector Pro, KeyHost, and E-Blaster are not detected as malware.
Bypassing anti-virus: Matousec.com whitepaper KHOBE – 8.0 earthquake for Windows desktop security software describes how malware developers could bypass anti-virus software. Using a Kernel Hook Bypassing Engine (KHOBE), researchers demonstrated how malicious code could be executed without anti-virus software scanning. Anti-virus software inserts itself in the System Service Descriptor Table (SSDT) to scan code before it is executed, but unscanned code can be loaded, swapped in and executed.
Note that this approach requires additional malware to install the malware which bypasses anti-virus protection. This emphasizes the need to:
- pursue all detected malware incidents for undetected malware payloads,
- inventory systems for anomalous services, software and system settings, and
- monitor for “phone home” activity from command and control botnets.
Follow up May 25, 2010 at KHOBE attack technique fails to gain major security threat status by Robert Westervelt, indicates that kernel hooks have been a treat channel for a long time. Once the kernel hook bypassing engine is installed, many compromises are possible.
Motivation: Presentations, such as Reality Check: Emerging Information Security Threats, show the lengths that attackers will go to in an effort to get money. Malware is often employed. It is evident that anti-virus software alone is not sufficient, but it is also evident that anti-virus software is always part of your defenses.
The “Is Anti-Virus Dead?” speculation relies upon expectations that are not shared. If the expectation is that anti-virus software using pattern matching (including heuristics) should always be a sufficient defense, that it should always detect new malware, then that expectation will not be met. Instead, anti-virus software should be supplemented with other measures.
For example, the new malware variations do not reflect new vulnerabilities. The new malware variations are attempts to avoid existing pattern detection of previous exploits. If the vulnerabilities have been mitigated (typically by patching), then the new and undetected malware variations that anti-virus software does not yet detect have no effect.
Bear in mind your preventative and defensive measures:
- Patch vulnerabilities.
- Use anti-virus software with pattern matching technology to detect known exploits of vulnerabilities, even those you have patched. Prevent the exploits from executing, even those that will fail because the vulnerability has been patched.
- Block access to known malware distributors.
- Remove unnecessary services and ports.
Supplement these preventative measures with reactive discovery measures.
- Damballa Failsafesensors, for example, analyze:
- DNS query behavior. Is the asset issuing an unusual number of domain look-ups that do not resolve to an IP address (NXDomains)? This is a popular technique criminals use to hide the command-and-control servers, and renders ‘block lists’ useless.
- Destination reputation. Is the location the asset is trying to connect to suspicious?
- Connection behaviors. Is the destination of the communications suspicious? Have connection attempts been successful?
- Automation. Does the query/connection behavior act like a user or seem more automated like it would be software-driven?
- Malware downloads. Have suspicious binaries been downloaded by this asset?
- Use behavioral analysis technology to (among other things) detect unknown exploits of unpatched vulnerabilities. See Using Behavioral Analysis To Discover Undetected Malware.
- Use analysis technologies that do not require behavioral analysis, such as those described in “What’s Different About This Approach?” to detect unknown exploits of unpatched vulnerabilities.
Do not confuse the preventative role of pattern matching anti-virus technology with the reactive role of behavioral analysis. Use what you learn from reactive approaches to enhance preventative approaches (learn from your mistakes).
Restore a compromised system to a trustworthy state by reimaging it or using one of the methods described in “Alternatives To Reimaging“. Specifically, do not rely upon virus cleaning measures, for reasons described in “Can You Clean a Virus?“.
The Top 10 Enterprise Botnets quotes Gunter Ollmann, vice president of research for botnet protection vendor Damballa:
“Over 60 percent of the malware we see associated with botnets has no AV signatures,” Ollmann says. “The real damage to enterprises is what’s outside the Top 10. But they still haven’t managed to overcome the big, noisy, [and known] threats that account for 80 percent of all detections.”
18 Feb 2010 in SearchSecurity: Zeus Trojan continues reign infecting 74,000 PCs in global botnet. Alex Cox, a principal analyst at NetWitness, reports:
the Zeus Trojan variant used in the latest attacks had a detection rate of less than 10% among antivirus software. The botnet communication was also shielded from detection by existing intrusion detection systems.
“This is not about a single piece of malware on 75,000 machines, it’s about how bad the security industry is responding to these incidents and how bad the problem is,” Cox said.
The cybercriminals exploited vulnerabilities in Adobe Flash as well as holes in Adobe Reader and Acrobat using malicious PDF applications in spear phishing attacks, according to Cox. They also used exploit kits to set up drive-by attacks to infect victims.
5 Mar 2012 in Infosecurity:
The criticism against anti-virus software is that it is very easy to get passed the first level of defense, signature recognition, by automatically and almost daily altering the malware’s signature.
But the danger in such comments is that the man or business in the street might start thinking that anti-virus, as one of the layers within a layered security defense, is no longer necessary or useful.
December 2012 Imperva Assessing the Effectiveness of Antivirus Solutions [pdf]:
1. The initial detection rate of a newly created virus is less than 5%. Although vendors try to update their detection
mechanisms, the initial detection rate of new viruses is nearly zero. We believe that the majority of antivirus products on the
market can’t keep up with the rate of virus propagation on the Internet.
2. For certain antivirus vendors, it may take up to four weeks to detect a new virus from the time of the initial scan.
3. The vendors with the best detection capabilities include those with free antivirus packages, Avast and Emisoft,
though they do have a high false positive rate.
How do you unearth the unknown variants? By treating the known detections as incidents. PICERL
See also: GrAVity: A Massively Parallel Antivirus Engine (improving pattern-matching performance using the processing power of the graphics adapter)
[…] Analysis To Discover Undetected Malware This post exists to flesh-out an outline in “Is Anti-Virus Dead?” The […]
[…] Anti-Virus: Note the changed role of Anti-Virus. Its role in Prevention and Recovery is diminished, but its role in Detection is (or should be) more significant. Not “dead“. […]
Excellent post. Thanks.
[…] also: Can You Clean a Virus?, Basic Virus Defense, Is Anti-Virus Dead?, The Anti-Virus Guy, Anti-Virus Remains […]