AV solution comparison: testing procedures

Qnetlabs takes a somewhat different approach than the traditional AV testing methods. First off, we only look into viruses that try and propagate themselves across networks. That means we specifically don't include e-mail viruses, IM viruses and malicious websites. Secondly, we don't feed a large amount of old, well-known, viruses into the AV solutions, but instead focus primarily on new threats we observe in our vast honeypot network. This leads to lower detection percentages than one would usually see in AV testing. Below, we'll outline two testing methods we think are important, if perhaps somewhat unconventional. The results of these testing methods are also displayed on the right hand side of this page (Top 3's).

Quick Response

The Quick Response indicates the percentage of recent (first seen in the last two months) verified malware an AV solution detects within a week of the verified malware first being spotted in our honeypot network. If malware is being spotted more frequently, its proper detection carries a stronger weight towards reaching a higher detection percentage than a one-off malware sighting/detection does.

The Quick Response aims to show which AV solutions are quick off the mark in detecting new threats.

Current Threat

The Current Threat indicates the percentage of verified malware, that tried to do actual infection attempts in the last two weeks, which has been detected rightaway by the AV solution.

The Current Threat aims to show which AV solutions are performing well when it comes to dealing with frequently seen, recent threats, mimicking the current real-world situation.

Terminology used

  • Verified malware: malware that has actually connected to a source on the Internet to download a payload or spread itself, or connect to a botnet controller
  • Honeypot network: the Quarantainenet Group's vast network of reactive honeypots
  • AV solution: The antivirus software we test the verified malware against. This software is running with automatic updates switched on, comparable to a home user situation
  • Infection Attempt: a 100% certain infection attempt by verified malware