Back to Article List
Anti Virus Software: Norton, McAfee, Trend Micro, or Hauri?
By Brien Posey
March 15, 2003
During the last several years, viruses have become increasingly more sophisticated. At the same time, the Internetís ever growing popularity and the steady adoption of ďalways on?broadband technologies have allowed viruses to spread quickly. Now more than ever, it is important to defend every computer in an organization against viruses in the most effective manner possible. The problem is that there are a number of available anti virus products available, each with their own strengths and weaknesses. To determine which anti virus product is the most effective, weíve tested four leading anti virus products against each other. In this paper, we will describe our testing methods and will present you with the results of each test.
Types of Anti Virus Software
For our test, we are comparing anti virus software from Norton (Symantec), McAfee, Trend Micro, and Hauri. Each of these companies manufacturers multiple anti virus products, each intended for different purposes. For the tests described in this paper, we chose to use the version intended for desktop computers. This means that our test results are valid for large businesses, small businesses, and for home users alike. The tests were conducted on January 22, 2003. During these tests, the latest virus definition files were downloaded for each product. We have made every effort to present our testing methods and results clearly. You can see the actual test screen captures by clicking on the various thumbnails found throughout this white paper. Links to each company's individual Web site are located at the end of this paper.
Product Being Tested
At the time that this paper was written, there were four major players in the anti virus. This list includes Norton, McAfee, Trend Micro, and Hauri. The comparisons are made in a random order and donít reflect our preferences or test results.
Test 1: Basic Detection and Repair
For our first test, we placed seven infected files onto a test PC. The files were infected with viruses such as Nimda, Klez, and Fun Love. The idea behind the test was simply to determine each productís effectiveness at detecting and cleaning common viruses. For this test, we disabled each anti virus programís automatic scanning engine, copied the infected files to a folder, and then manually scanned the system.
We began our tests using McAfee. McAfee had no trouble detecting our infected files. As you can see in Figure A, after detecting the infected files, McAfee asks the user to clean the infected files, and if the clean fails, to delete the files.
Figure A: McAfee detects the infection and asks the end user to take action.
The end result was that McAfee was able to detect all seven of the infections, but was unable to clean any of them. You can see the test results in Figure B.
Figure B: McAfee detected all 7 infected files but was unable to clean any of them.
Next, we placed the same seven infected files onto the test machine and used Norton Antivirus try to detect and repair them. As you can see in Figure C, Norton claimed to detect nine infections, even though only seven files were actually infected.
Figure C: Norton AntiVirus detected nine infections instead of seven.
Like McAfee, Norton AntiVirus asked the user to click a button to begin the repair process. After clicking the Repair button, Norton reported that it was unable to repair any of the seven files. Norton then recommended that the files be quarantined. You can see these test results in Figures D and E.
Figure D: Norton AntiVirus detected nine infected files instead of seven, and was unable to repair any of them.
Figure E: Norton AntiVirus was unable to repair the infected files.
Next, we attempted to detect and repair the same nine infections using Trend Microís PC-cillin. As you can see in Figure F, the Trend Micro product detected seven infections, and was unable to clean any of them. The Trend Micro product automatically quarantined the files that it was unable to clean.
Figure F: Trend Microís PC-cillin detected seven infections and quarantined them rather than cleaning them.
Finally, we tested Hauriís ViRobot Expert. ViRobot was able to detect all seven infections and was able to repair them faster than we could blink. You can see these test results in Figure G.
Figure G: Hauriís ViRobot detected all seven viruses and was able to repair them automatically.
Test 2: Integrity of Repaired Files
In Test 1, Hauriís ViRobot was the only anti virus program that was able to repair the infected files. For our second test, we acquired some infected files that any anti virus program should be able to repair. We infected a system DLL file and a system level executable with Nimda, and Fun Love respectively. As you can see in Figure H, we began with a file named CreateCDDA.DLL and a file named WIN32F~3.EXE. In Figure H, pay close attention to the file sizes, date/time stamp, and to the fileís icons.
Figure H: Pay close attention to the file names, icons, sizes, and date/time stamps.
We began this test by running McAfee against the two infected files. Upon doing so, McAfee detected the virus and prompted us for what action to take. We clicked the Clean button, and McAfee reported that the files had been cleaned, as shown in Figure I. However, if you look at the files in the figure, youíll notice that the file sizes have changed. This is normal since viral code has been deleted from the file. Youíll also notice though that the date/time stamp has changed and that the WIN32F~3.EXE fileís icon has changed to a generic icon.
At first, having an altered date/time stamp and an altered icon may not seem like a big deal. However, itís very important to preserve date/time stamps. For example, many times when you contact Microsoft for Technical support, they will ask you for the date/time stamp on various system files, in order to determine the fileís version. If the date/time stamp has been altered, itís impossible to tell at a glance if the file is the correct version. Likewise, if an executable fileís icon has changed, it could possibly mean that the file has lost some of its integrity, and that more may have been removed than just viral code.
Figure I: McAfee cleaned the files, but altered the date/time stamp and the icons.
Next, we tried to disinfect the same set of viruses using Nortonís. Nortonís detected the infection with no problems. When we clicked the Repair button, We received a message that the infected DLL file was repaired, but that the repair failed on the WIN32F~3.EXE file, which was infected with Fun Love. The strange thing about this is that the CreateCDDA.DLL file was infected with Nimda. Nimda is basically a virus that built on Fun Love. Therefore, it seems strange that Nortonís could fix Nimda, but not Fun Love. You can see the test results in Figure J. After Nortonís completed, the fileís icons were preserved, but the date/time stamp was reset, even for the file that couldnít be repaired. You can see this in Figure K.
Figure J: Nortonís cleaned Nimda, but not Fun Love.
Figure K: Nortonís reset the fileís date/time stamp.
After completing our testing with Nortonís, we tested PC-cillin. The Trend Micro product detected four viruses even though there were only two files. As you can see in Figure L, PC-cillin misidentified the viruses and simply quarantined the viruses.
Figure L: PC-cillin misidentified and miscounted the viruses.
Finally, we tested Hauriís ViRobot against the same two infected files. As you can see in Figure M, ViRobot not only repaired the infected files, but also managed to preserve the date/time stamp and the icon. We should point out though that while the results were very obvious with the other three products, we had to perform a manual screen refresh by pressing F5 to see what ViRobot had done with the files.
Figure M: ViRobot repaired the files and left the icons and date/time stamps intact.
Pittsburgh Steelers Women's Keynote Layered Tank Top - White,New York Jets 24" X 39" Coir Door Mat,Buffalo Bills 32oz. Stainless Steel Keeper Tumbler with Lid.Women's New Orleans Saints Gold Extra Pointed Quilted Jacket,Men's Pittsburgh Steelers Ben Roethlisberger Nike White Limited Jersey,Womens Kansas City Chiefs Majestic Red Coin Toss IV Long Sleeve T-Shirt Bryce Petty Jersey Cheap.Men's Green Bay Packers Black End Around Pullover Hoodie,Baltimore Ravens 3-Pack Memo Notebook.Women's Pittsburgh Steelers Majestic Black Play For Me T-Shirt,Girls Toddler Dallas Cowboys New Era Pink Tye Dye Fan 9TWENTY Adjustable Hat,Mens New York Giants Nike Royal Blue Ticket Long Sleeve T-Shirt Ryan Fitzpatrick For Sale Cheap.Mens Buffalo Bills New Era Royal Blue 39THIRTY Team Classic Flex Hat,Women's Seattle Seahawks Cutter & Buck College Navy Vancouver Full Zip Sweatshirt.Touch by Alyssa Milano Minnesota Vikings Women's Morgan Tri-Blend T-Shirt - Purple/Gold,FanSided - Sports News, Entertainment, Lifestyle & Technology - 300+ Sites,Denver Broncos 16oz. Logo Striped Glass Latte Mug.Minnesota Vikings Dream Lites Pillow Pet,Women's Buffalo Bills Junk Food Royal Champion Fleece Sweatshirt,Men's Houston Texans Majestic Navy Hot Read Full-Zip Hoodie New York Jets Jerseys for Sale.New York Giants Stadium Panoramic Photomint,Pro Line Tennessee Titans Heritage Football Jersey Long Sleeve T-Shirt - Navy Blue Cheap-New-York-Giants-Jets-JerseysTest 3: Detecting Viruses In Memory
When we approached Hauri about our tests, they claimed that their ViRobot products could actually detect viruses in memory and could even clean individual executing processes. Hauri provided us with a utility that is designed to test a systemís memory for the existence of the Klez virus. Because this utility is a closely guarded trade secret, our non disclosure agreement with Hauri prevents us from revealing the name of the utility. In the screen shots that youíll see in this section, file names and commands have been blurred, for legal reasons. However, the screen shots have not been doctored in any other way.
Needless to say, we were immediately suspicious of this utility since it was provided to us by one of the anti virus manufacturers under such secrecy. However, rigorous independent testing has confirmed that the utility that weíll be using for Test 3 is indeed trustworthy.
For this test, we used the SQL Server client configuration utility as a test executable. For the test, we infected the utility with the Klez virus. For the test, we verified that the virus was not present in memory, ran the executable to infect the system, verified that the virus was present in memory, disinfected the virus, and then checked the systemís memory to see if the memory was still infected. You can see an example of this process shown in Figure N. In this figure, you can see where we tested the memory, infected the system by loading CLICONFG, and tested the memory again.
Figure N: This is how we test to see if a systemís memory is infected.
We began by testing McAfee. When McAfee ran, it detected the virus right away. McAfee then closed the infected process (The SQL client configuration utility), and then reported that the system was clean. However, as you can see in Figure O, the systemís memory was still infected.
Figure O: The systemís memory was still infected, even after the virus was cleaned by McAfee.
Next, we repeated the test with Norton AntiVirus. Norton AntiVirus detected the virus, but was unable to repair it. Norton left the infected process, and the systemís memory remained infected, as shown in Figure P.
Figure P: Norton was unable to disinfect the virus.
For the next test, we attempted to scan for the virus with PC-cillin. As you can see in Figure Q, PC-cillin detected the virus, but was unable to clean it. The virus was also still present in memory, and the infected process continued to run.
Figure Q: PC-cillin detected the virus, but could not repair it.
Finally, we repeated the test using ViRobot. Like the other antivirus products, ViRobot had no trouble detecting the infection. However, ViRobot then displayed the message shown in Figure R. This message indicated that the infected file was presently running. ViRobot then gave us a chance to save any documents that might have been open, before closing the infected process.
Figure R: ViRobot detected the infection and asked to close the infected process.
After closing the infected process, ViRobot disinfected the file and the systemís memory, and then reopened the process. If you look at Figure S, you can see that the memory was completely clean after ViRobot finished cleaning the system.
Figure S: ViRobot was able to remove the infection from memory.
Test 4: Performance
For our final test, we wanted to benchmark each productís performance during a full system scan. We performed this test because the more processor time that a product uses during a system scan, the less responsive that the PC will be. Therefore, we were checking to see which product has the lowest processor utilization.
For this series of tests, we closed all running applications except for the product that we were testing. We then initiated a full system scan, and opened the Windows Task Manager to watch the Performance tab. We waited until a representative amount of activity had occurred prior to taking the screen shots.
We began the process by performing a full system scan with McAfee. As you can see in Figure T, although there were spikes in the processor utilization level, McAfeeís overall CPU utilization was relatively low, averaging around 30%.
Figure T: McAfee had around a 30% CPU utilization.
Next, we repeated the same test using Norton AntiVirus. As you can see in Figure U, Norton AntiVirus had nearly a 100% CPU utilization during the scanning process. The period of low activity that you see in the figure prior to the heavy activity was generated by us simply loading the Norton AntiVirus console The full system scan began at the point in the graph where the activity increased so dramatically.
Figure U: Norton AntiVirus held the processor at near 100% utilization during the scan.
At this point, we tested PC-cillin. PC-cillin performed very well in the processor utilization test. The average processor utilization was well under 20%, as shown in Figure V.
Figure V: PC-cillin had very low processor overhead.
For our final test, we measured the processor utilization while ViRobot was scanning the system. As you can see in Figure W, ViRobot sustained an extremely low level of activity, well below 10% CPU utilization. The spikes that you see in CPU activity at the beginning of this chart were from when we loaded the ViRobot console.
Figure W: ViRobot had extremely low CPU usage.
Although the table below outlines the good and bad points of each product, determining the best product isnít as simple as counting to see which product has the most points. The reason for this is that some features are more important than others and therefore disserve stronger consideration. The chart below is a weighted comparison of the products based on which features are the most important. In this analysis, each product has been given between one and five points for each area of comparison, with five representing the highest possible score. The productís score in each area is multiplied by its weight to determine the total number of points for the feature. At the end, all of the points are tallied together to determine the results.
As you can see from the charts above, our absolute favorite product was newcomer Hauriís ViRobot, which earned a perfect score. Our second favorite was McAfee, followed by Trend Micro, with Norton in last place. As you view these results, remember that Relevant Technologies is an independent security research firm, and that we have provided screen shots of the actual tests, to validate our findings. If you would like more information about any of the products that we have discussed in this paper, you can contact each respective company via their Web site. The addresses are as follows:
Trend Micro: www.trendmicro.com
|Copyright 1997-2015 Relevant Technologies. All rights reserved | Legal and Privacy | Sitemap
Email: firstname.lastname@example.org | Tel: 240.786.4858 | Fax: 855.451.5466 | 8160 Maple Lawn Blvd, Suite 200, Fulton, MD 20759