AV-Comparatives: Performance Test

Last updated on 11 Dec 2011, Sun

*Users are encouraged to try out the software on their own PC’s and form an opinion based on their own observations.

(Performance Test)

Tested products:

The following products were evaluated (with default settings) in this test (November 2011):

avast! Free Antivirus 6.0

AVG Anti-Virus 2012

AVIRA Free Antivirus 2012

Bitdefender Antivirus Plus 2012

eScan Anti-Virus 11

ESET NOD32 Antivirus 5

F-Secure Anti-Virus 2012

G DATA AntiVirus 2012

K7 Antivirus Plus 11.1

Kaspersky Anti-Virus 2012

McAfee AntiVirus Plus 2012

Microsoft Security Essentials 2.1

Panda Cloud Antivirus Free 1.5.1

PC Tools Spyware Doctor with Antivirus 2012

Qihoo 360 Antivirus 2.0

Sophos Endpoint Security 9.7

Symantec Norton AntiVirus 2012

Trend Micro Titanium Antivirus Plus 2012

TrustPort Antivirus 2012

Webroot SecureAnywhere Antivirus 2012

Please note that the results in this report apply only to the products/versions listed above (e.g. 64-Bit versions, product version, etc.). Also, keep in mind that different vendors offer different (and differing quantities of) features in their products.

The following activities/tests were performed under Windows 7 Professional SP1 64-Bit:

• File copying
• Archiving / Unarchiving
• Encoding / Transcoding
• Installing / Uninstalling applications
• Launching applications
• Downloading files
• PC Mark 7 Professional Testing Suite

(Click on the image for a clearer view)

I converted the above report into excel so that is easier to "read".

(Click on the image for a clearer view)

PC Mark Tests

In order to provide an industry-recognized performance test, we used the PC Mark 7 Professional Edition4 testing suite.

Users using PC Mark 7 should take care to minimize all external factors which could affect the testing suite and follow strictly at least the considerations/suggestions documented inside the PC Mark manual, in order to get consistent and valid/useful results.

Furthermore, the tests should be repeated several times to verify them. For more information about the various consumer scenarios tests included in PC Mark, please read the whitepaper on their website5. “Without AV” is tested on a baseline6 system which scores 2024 in the PC Mark test.

Summarized results

Users should weight the various subtests according to their needs. We applied a scoring system in order to sum up the various results.

For “file copying” we took the mean values, as well as for “launching applications” (on subsequent runs). Like in previous performance reports, “very fast” gets 15 points, “fast” gets 10 points, “mediocre” gets 5 points and “slow” gets zero points. This leads to the following results:

Overall for Performance Test
Top Anti-Virus with the lowest impact on overall result are Eset & K7.


Reference:
http://www.av-comparatives.org/

Read More

Av-Comparatives - Anti-Phishing Tests

All Contents | Last updated on 30Oct2011, Sunday

Anti-Phishing Tests - October 2011

Note:
These tests are not conducted by Chimalogy but by Av-Comparatives.

Test procedure

In our common test scenario we simulate a user which relies on the Anti-Phishing protection provided by its security product while browsing the web (and/or checking his web mail

e-mail account, i.e. anti-spam features are not considered as they are not the scope of this test).

The test was done using Windows XP SP3 and Internet Explorer 7 (without the build-in phishing blocker in order to get browser-independent results) under VMware. All security products were tested using default settings. All products were tested in parallel at the same time on the same URLs.

Tested products (19)

• AVG Internet Security 2011

• Avira Premium Security Suite 10.1

• Bitdefender Internet Security 2012

• Bullguard Internet Security 10.0

• eScan Internet Security 11.0

• ESET Smart Security 4.2

• F-Secure Internet Security 2011

• G DATA Internet Security 2012

• K7 Total Security 11.1

• Kaspersky Internet Security 2012

• McAfee Total Protection 2011

• Panda Internet Security 2012

• PC Tools Internet Security 2011

• Qihoo 360 Internet Security 2.0

• Quick Heal Internet Security 2011

• Symantec Norton Internet Security 2011

• Trend Micro Titanium Internet Security 2012

• Trustport Internet Security 2012

• Webroot Internet Security Complete 7.0

We were asked to test also Avast, Microsoft (Security Essentials) and Sophos. But Avast and Sophos do not have any Anti-Phishing feature, except blocking phishing mails with their spam filter.

Microsoft does block phishing websites in its browser (Microsoft Internet Explorer) and its spam filter (e.g. Microsoft Outlook); Security Essentials is not designed for this task.

ESET doesn’t have a dedicated module for anti-phishing protection yet. However, phishing sites are currently blocked along with other potentially harmful sites (like in most other products), employing various technologies such as web filter, parental control and detection of suspicious content aimed at the phishing landing pages themselves.

Qihoo is mainly targeted to block Chinese phishing sites.

Test Set

Phishing URLs taken out from spammed prevalent phishing mails and/or collected by our crawler and researchers. All phishing URLs had to be active/online and attempt to get personal information. Duplicate phishing URLs, as well as phishing campaigns hosted on same server/IP were removed.

All test cases were manually re-verified after the test. At the end, only 697 different and valid Phishing URLs remained. The phishing campaigns targeted various types of personal data.

Among those were (in the following order) phishing attempts to gather e.g. login credentials etc. for: PayPal, eBay, Online Banking & Credit cards, Social networks, Online Games,

E-mail accounts and other online services.

Below you see the percentages of blocked phishing websites (size of test set: 697 phishing URLs). Please take into consideration the false alarm rates (on next pages) when looking at the below results (products with false alarms are marked with an asterisk).

(Click on the image for a clearer view)


(Click on the image for a clearer view)

Anti-Phishing “False Alarm” Test

For the Anti-Phishing False Alarm Test we selected 1000 legitimate banking sites (all of them using HTTPS) from all over the world and checked if those legitimate online banking sites were blocked by the various security products.

Wrongly blocking such sites is a serious mistake. From the tested 19 products, only two (K7 and Webroot) had false alarms on the tested 1000 legitimate online banking sites:

K7 (1 false alarm):

• Bank Islam from Malaysia

Webroot (12 false alarms):

• AB Finance Bank from Russia

• Bank of Kenya from Kenya

• BLC Bank from Lebanon

• Dawia Next Bank from Japan

• Dhanlaxmi Bank from India

• Habib Bank Zuricj from UAE

• ING NL from Netherlands

• Marfin Laiki Bank from Cyprus

• Refah Bank from Israel

• Saxo eBank from Sweden

• ScotiaBank from Mexico

The discovered false alarms have been reported to the respective vendors and are now no longer blocked.

Which products support which browser?

Most browsers already include their own Anti-Phishing technologies. Nevertheless, due to the complexity and amount of social engineered scams and phishing attempts, it is recommended to make use of the Anti-Phishing features provided by security products too.

Practically all security products support the main stream browsers (i.e. Internet Explorer and Firefox), while only few support also other browsers. Due to that, it may be safer to make use of a supported browser to perform e.g. online banking.

(Click on the image for a clearer view)

Passed - Green (Supports all of the 5 commonly used browsers)

Neutral - Orange (Supports at least 3 commmonly used browsers)

Failed - Red (Supports less than 3 commonly used browsers)

Passed:

AVIRA Premium Security Suite

Bitdefender Internet Security

ESET Smart Security

G DATA Internet Security

Kaspersky Internet Security

Panda Internet Security

Trend Micro Titanium Internet Security

Neutral:

PC Tools Internet Security

Symantec Norton Internet Security

Failed:

AVG Internet Security

Bullguard Internet Security

eScan Internet Security

F-Secure Internet Security

K7 Total Security

McAfee Total Protection

Qihoo 360 Internet Security

Quick Heal Internet Security

Trustport Internet Security

Webroot Internet Security

Conclusion:

Top 7 that passes the overall tests.

1) 94.4% - Bitdefender

2) 92.8% -Trend Micro

3) 89.1% - AVIRA

4) 88.5% - G DATA

5) 85.1% - Kaspersky

6) 84.5% - Panda

7) 78.0% - ESET

Note:

I'm suprised that ESET managed to pass since it doesn’t have a dedicated module for anti-phishing protection yet but phishing sites is currently blocked along with other potentially harmful sites.

Chance of getting phished = Your Common Sense + Browsers + Security Products

In fact, this test seems pretty useless(except is show where is your anti-virus standing) to me as common sense play the largest part in this.

My advise is to always keep your software like browser and security product up-to-date and never open/click on any suspicious links from unknown and known person. Always double check with the person before clicking the link and is possible, type the link in your browser instead of clicking it.

Reference:

http://www.av-comparatives.org/en/comparativesreviews/anti-phishing-test

All Contents

Read More

AV-Comparatives Tests

Last update on 30 Aug 2011, Sunday

Whole Product "Real-World" Dynamic Test

Settings

We use every security suite with its default (out-of-the-box) settings. If user interactions are required, we will choose the default option.

Our whole-product dynamic test aims to simulate real-world conditions as experienced every day by users. Therefore, if there is no predefined action, we will always use the same action where we consider the warning/message to be very clear and definitive.

If the message leaves it up to the user, we will mark it as such, and if the message is very vague, misleading or even suggests trusting e.g. the malicious file/URL/behaviour, we will consider it to be a miss, as the ordinary user would.

This year we will be stricter with required user decisions/interactions than last year. We consider “protection” to mean that the system is not compromised.

This means that the malware is not running (or is removed/terminated) and there are no significant/malicious system changes.

An outbound firewall alert about a running malware process, which asks whether or not to block traffic form the users’ workstation to the internet is too little, too late and not considered by us to be protection.

For more details on how they test the anti-virus, refer to their AV-Comparatives website.

Tested products

The following products were tested in the official Whole-Product Dynamic main test series.

In this type of test we usually include Internet Security Suites, although also other product versions fit, because what is tested is the “protection” provided by the various products against a set of real-world threats.

Main product versions used for the monthly test-runs:

Whole Product Dynamic "Real World" Test - Graph Bar

Note:

Blocked(Green) / User Dependent(Yellow) /Compromised(Red) - [All in percentage %]

Results:

01) F-SECURE: 99.2 / 0.2 / 0.6

02) SYMANTEC: 99.1 / 0.4 / 0.4

03) BITDEFENDER: 99.1 / 0 / 0.9

04) G DATA: 98.9 / 0 / 1.1

05) TREND MICRO: 98.6 / 0 / 1.4

06) PANDA: 98.6 / 0 / 1.4

07) ESET: 98.2 / 0 / 1.8

08) KASPERSKY: 97.7 / 0.9 / 1.3

09) AVAST: 97.1 / 1.3 / 1.6

10) AVIRA: 96.9 / 0 / 3.1

11) QIHOO: 96.1 / 1.3 / 2.6

12) SOPHOS: 95.6 / 0 / 4.4

13) AVG: 95.1 / 0.2 / 4.7

14) WEBROOT: 94.7 / 0 / 5.3

15) MCAFEE: 93.5 / 0 / 6.5

16) K7: 91.8 / 0.7 / 7.5

17) PC TOOLS: 91.2 / 6.7 / 2.1

Reading according to graph, is slightly different from the result is given in the table below.



Top 7: F-Secure, Symantec, Bitdefender, G Data, Trend Micro, Panda, ESET

Whole-Product “False Alarm” Test (wrongly blocked domains/files)

The false alarm test in the Whole-Product Dynamic test consists of two parts: wrongly blocked domains (while browsing) and wrongly blocked files (while downloading/installing).



Top 3: AVG, BitDefender, ESET

Overall for Whole Product "Real-World" Dynamic Test
Bitdefender and ESET have 1 of the highest detection rate and lowest false alarm.

Overall Test for Performance test and "Real-World" Dynamic Test:
ESET is chosen as it is the top 2 overall for Performance Test and has the 1 of the lowest false alarm and 1 of the highest detection rate for "Real-World" Dynamic Test.


Reference:

http://www.av-comparatives.org/

All Contents

Read More