Each antivirus program is installed on its own physical PC, and update the signatures. The malware sample files are then scanned using the program’s standard scanning procedure, and the number of detections is recorded. The PCs remain connected to the Internet during the test, meaning that the security programs can use any cloud features provided by their manufacturers.

Typically, more than 100, 000 malware samples are used for the test. These are prevalent malicious files of all types that have been recently collected, i.e. within a period of a few weeks or months before the test is performed.

To ensure that the tested programs do not simply identify all unknown files as malware, a false-positive test is also conducted for every File Detection Test. Any programs with a high rate of false positives will have their award reduced by at least one level.

The File Detection Test assesses the ability of antivirus programs to detect malicious files on a system. Whilst the test only assesses one antimalware feature of the programs, this feature is important for a security solution. This is because it can identify malware attacks from sources other than the Internet, and detect malicious files already present on the system.

The graph below shows the test results against "out-of-box" malware detection provided by Microsoft Windows. In Windows 8, this is provided by Windows Defender, which is pre-installed by default with the operating system. The equivalent in Windows 7 is Microsoft Security Essentials, which is not pre-installed, but can easily be added for free as an option via the Windows Update service. We use this to compare the value of this feature – i.e. static malware detection - provided to users of third-party Anti-Virus solutions.

In order to better evaluate the quality of the file detection capabilities (distinguish good files from malicious files) of anti-virus products, we provide a false alarm test. False alarms can sometimes cause as much trouble as a real infection. Please consider the false alarm rate when looking at the detection rates, as a product which is prone to false alarms achieves higher detection rates more easily. The lower the false positives means that the antivirus’ performance is very solid.

Source: www.lavasoft.com