2018 Anti-Virus comparison

AV-Comparatives logo

A breakdown of the performance of AntiVirus over the past year, based on the Real World Protection Test results from
AV-Comparatives
The top 5 have been marked in green in each image, and the baseline of Microsoft AV is selected in each for comparison.
I have broken it down into how often each package from the monthly top 5 features in the top 5, and how many times a package from this group has the worst hit rate for false positives (marked with *).

First the 2 half year aggregate results.

February to June 2018

1 Trend Micro
2 F-Secure
3 Bitdefender
4 Kaspersky Lab
5 Avira
16 Microsoft

July to November

1 Bitdefender
2 F-Secure
3 Trend Micro
4 Avira
5 Avast
14 Microsoft

Individual months


February
1 Kaspersky
2 Trend Micro
3 Microsoft
4 F-Secure *
5 Symantec


March
1 Panda
2 Trend Micro
3 F-Secure *
4 Symantec
5 Bitdefender
17 Microsoft


April
1 Avira
2 McAfee
3 Bitdefender
4 Panda
5 Trend Micro
18 Microsoft


May
1 Avira
2 Bitdefender
3 Tencent
4 Microsoft
5 Trend Micro


June
1 Bitdefender
2 McAfee
3 Avast
4 AVG
5 Kaspersky
16 Microsoft


July
1 Kaspersky
2 Avira
3 Bitdefender
4 Tencent
5 Quick Heal
8 Microsoft *


August
1 Bitdefender
2 Kaspersky
3 Avira
4 F-Secure
5 McAfee
10 Microsoft *


September
1 Avast
2 AVG
3 Bitdefender
4 VIPRE
5 F-Secure
7 Microsoft *


October
1 Bitdefender
2 F-Secure
3 Trend Micro
4 Panda
5 Microsoft *


November
1 Tencent
2 Avast
3 AVG
4 Avira
5 BullGuard
18 Microsoft *

Final Analysis and  Context

Months in the top 5

Bitdefender x8
Avira x5
F-Secure x5
Trend Micro x5
Kaspersky x4
Avast x3
AVG x3
McAfee x3
Microsoft x3
Panda x3
Tencent x3
BullGuard x1
Quick Heal x1
VIPRE x1

Highest false positives

F-Secure x5
Microsoft x5

Microsoft (the baseline) are rarely in the top 5 in any year, and are mostly near the bottom.
Last year they managed 3 months in the top 5, coming equal with Avast, AVG, McAfee, Panda and Tencent.

Microsoft share the most amount of false positives with F-Secure at 5 months each.
Both out-do the consistently worst performing AV on this score, however F-Secure is also in the top 5 for an equal amount of time.

Avira, F-Secure and Trend Micro all spent 5 months in the top 5, but all showed occasional big swings in reliability, however they most of the time stayed in the top half even when not in the top 5.

Kaspersky may only have been in the top 5 for 4 months, but only twice dropped to the bottom half, and consistently stayed near the top.

There is a clear winner here, and that has to be Bitdefender, which yet again stayed in the top 5 longer than any other with an impressive 8 months, and always with a low false positive rate.

Short recommendations:
If you want to pay, buy Bitdefender. (the free version is very limited)
If you want free, also consider Avira and Kaspersky Lab
If you want a lucky 8-ball, use Microsoft

Comment and criticism of this post in the forums
https://forum.vivaldi.net/topic/34007/2018-av-comparison

Ageing techno-hippy armed with a radio show and not afraid to use it.

5 comments Write a comment

  1. Thanks for a great, concise review. Of course, detection rate and system peformance impact aren’t the only important factors here. I think there are a few other issues worth explicit mention here:

    System stability: An AV with a stellar detection rate will be useless if it is unstable and causes your system to lock-up, freeze for periods, or outright-crash. Theoretically, the clear winner here will be Microsoft, because they are the authors of the operating-system their AV runs on – and in fact, in Windows 10 it’s a baked-in part of the OS.

    User control: False-positives happen. Bitdefender famously bricked numerous PCs about 10 years ago with the “Trojan.FakeAlert” false-positive fiasco. The faulty virus signature triggered an auto-quarantine for any 64-bit executable or dll. Other AVs have done the same over the years, so I don’t want to single-out Bitdefender, but it highlights a very important issue. Keeping your files safe doesn’t just mean keeping them out of the hands of hackers, it also means keeping them IN your hands! The fact that a system is virus-free is irrelevant when it’s corrupt and won’t boot… If an AV doesn’t have the option to set it to “ask me” on detection instead of auto-quarantining, I don’t install it.

    Does it break HTTPS? A lot of AVs now perform a “man in the middle” attack on your web traffic, so they can scan your HTTPS traffic for viruses. Even if you trust your AV provider completely (you should do, otherwise you should pick another one), this is a very, very bad thing. This has unfortunately been demonstrated to significantly reduce your security. One AV provider used the same certificate for all installations, meaning anyone could create a phishing site and sign it with that AV provider’s certificate to make it seem legitimate. Most of these implementations prevent your browser from identifying invalid HTTPS certificates (e.g. if they’ve been revoked, expired or spoofed). HTTPS data should be indecipherable between the remote server and the web browser. If an AV can’t scan the traffic directly, it should monitor the browser and system’s behaviour instead. It shouldn’t reduce the system’s security so it can just scan the traffic anyway. One notable AV that provides this HTTPS scanning capability is Avast!, however I’m pleased to note that last time I looked at their software, it was still possible to disable this feature in the settings for their web shield.

    Trustworthiness of the company: Kaspersky is a good – if controversial – example here. Their detection-rate (and protection-rate in general) is excellent; however if you’re American then you potentially might not consider them trustworthy. (I’m not saying they’re not trustworthy – that’s something you’d have to evaluate for yourself). There are other more obvious examples, too. For instance, Chinese company Qihoo 360 has been caught cheating AV tests, has been accused by investigators of implementing backdoors/spying, enters into numerous lawsuits with competitors, and has a clear conflict of interest (main business is advertising). Panda has links to the Scientology cult. Sophos had a subdivision which sold spying technology to the Syrian government around 2010.

    I’m currently wondering which AV software to use. Most free solutions have all the basic features I want but have adverts in them, yet buying the full version gives you a piece of bloatware that is full of features I won’t ever use. I’d happily pay for Bitdefender if they’d just produce the free edition with an “ask me” option instead of auto-quarantine (and if it also didn’t break HTTPS). I’d pay to have Avira free with the ads removed – even if they did used to bundle the ask toolbar with their software. I really like F-Secure AV, as the entry-level basic AV is lightweight and functional: It’s nearly everything you could ask for in an AV, but again it doesn’t provide an “ask me” option.

  2. Dr.Flay™

    I do agree that stability and reliability are as important a factor as the level of protection, but the 1 and only argument that ever comes forward to defend MS AV solutions is that they don’t cause problems.
    By the same criteria I could say that running any another program that is poor at detecting malware is also a good choice because it never crashes corrupts the system.
    If an AV lacks features and abilities it will never run into problems with them. The less features it has the less problems, no features no problems.
    Most of the problem features such as HTTPS scanning have always been optional or require an extra browser extension so can easily be avoided if there is a concern.
    The irony of the few usual AV vendors with their certificate borking the cert validation, is that Comodo and Symantec were both certificate authorities who should know better, but have repeatedly been very stupid with certificates generally.

    Recently MS security was found to be being used as a way to bypass security and get into the system. Nicely demonstrating that all AV provide an extra vector of attack.

    It is a game of numbers here. You weigh the odds.
    Is it more likely you will come across malware your AV does not recognise or more likely your AV will eat itself or your system ?
    Statistically there is more chance of being bitten by malware than your AV, as those often legendary self destruction moments are thankfully rare and only happened to a few, which again tends to be the same few culprits.

    MS AV shows a regularly high amount of false detections. personally I see this inability to get it right, just as likely to bork someones system if they let it autofix everything. It may not kill MS components but many peoples systems rely on non MS software to function or do their job.
    How often have you personally seen MS security block valid malware, compared with how often you have to override it because it falsely detects a good file.
    I can’t see why people are so happy to sweep aside the failure to get it right, simply because it does not seem to have as many problems.
    People have used the same excuse with Norton and Mcafee for years “I have never seen any malware warnings so it must be good”.
    Cannot tell good from bad, also indicates cannot tell bad from good.
    That is why I offer the Magic 8 ball as the option. I can totally guarantee it will not crash your system or bork anything.

    When I get PCs to fix and clean from malware they are always “protected” by the same usual suspects. It is always Norton, McAfee, Avast or MS Defender.
    I have yet to have a system come in for cleaning that has 1 of the regular top 5 AV.
    “The proof of the pudding is in the eating” as they say, and MS Defender cannot find it’s own arse in a sandstorm most of the time.
    Both my main PCs are “protected” by MS security, and so far it is by using standalones, remote sandboxes and browser extensions that I have detected malware.
    The Windows 7 PC has MS EMET (Enhanced Mitigation Experience Tool) installed, as per the requirements to meet the MS baseline they use when quoting how good MS security is.
    Who do you know that;
    a) has heard of EMET ?
    b) has installed EMET and changed it from the default (opt-in) to high security (opt-out) ?
    c) only uses MS browsers ?
    d) has ever used the Baseline security scanner ?
    Without EMET installed to stop malware from doing damage, MS Defender is only half of what you need, and does not meet the baseline.
    Without browser integration users have no first-line protection from bad links, malvertising, and malware jscripts etc.
    MS have released an extension for other browsers, but all it does is check links against smartscreen, no actual malware detecting in the browser session.
    You can get the same level of link protection (maybe better) by using an adblocker with malware blocklists.
    I favour adding malware and malvertising sites to my HOSTS file so they are blocked for all software.

    It does not help when Microsoft are not very honest in their comparisons to other AV. They rely on EMET features set to high security, and an otherwise baseline secure system to beat malware infections, not just using Defender or MS Security Essentials.
    A big part of the problem will always be the curse of defaults and the fact most people never change them. I never liked the defaults in any AV yet.

    I tend to recommend Avira for most people that want a free option due to the amount of optional extras it has, so makes for a better all round security suite rather than just an AV. The browser integration is also optional.
    Yeah the popup ads in the tray are a bit annoying but at least they are only from Avira and for Avira products, and you get the option to not see that ad again.
    It is rare for a major bork with Avira due to the beta program. They make the full products available for free to anyone joining the beta site so have a good amount of feedback from the wild before changes move to the public.
    If you want to get rid of the popups, perhaps think about contributing some time and the occasional bug report. They are keen to get user experience feedback before making changes to the way it works, so often all they need is general feedback.

  3. Thank you for such a lengthy and thought-out reply. I wasn’t actually recommending Microsoft, despite the fact they’ve had a recent run of top detection rates by labs. Like you, their lack of consistency in performance makes me not consider them as an option. I sometimes occasionally run a scan as a second-opinion, but that’s it. Windows defender is the only AV I’ve never had a false-positive from. I didn’t realise they were supposed to be prone to them! I thought the lack of FPs was, in part due to its lack of general detection at all! I also never rely on Defender, simply because it’s a baked-in part of the system, therefore new malware will always target it either as an attack-vector or as “the one to avoid”. Sure, it might be able to detect old malware once it’s been added to the database, but it’s the one solution everybody has on their system – therefore I’d expect zero-day detection to be awful, as any malware worth its salt will specifically avoid detection by it. My personal favourite free AV, like you, is Avira. I have recommended it (and used it myself, on and off) since it was called “H+BEDV Antivir”, back in the 90s. I have never known it let anything past, and its false positive rate doesn’t seem to be as high as, say, Avast! or AVG. In terms of company ethics, I don’t put them too far behind the likes of F-Secure.

    Of course, I don’t really need antivirus for my usage. I have it because “you’re supposed to”. I keep a Windows installation solely to run the updater tool for my car’s ECU-tuner. It boots up for that, and that’s about it. No browsing, document editing, program-installing or even internet connectivity apart from that. My home computing is all GNU/Linux and at present, intrusion and rootkit-detection, network-monitoring, etc. are more relevant than traditional AV.

    • Dr.Flay™

      Lengthy is an understatement. I dumped a whole blog-worth into a reply.
      You kinda took the brunt of stuff I had going round my head due to repeat discussions (sorry).
      MS has shown a clear ability to be very capable, but I find the regular high rate of false detections too much of a worry.
      The very reasonable defence is that it is likely due to it being over cautious with unknown files.
      However this makes me think it is relying more on recognition than heuristics. I could use white-listing of everything myself if that is the Microsoft solution.

      I was pointed at this page yesterday as a defence of MS Defender.
      https://www.av-comparatives.org/tests/real-world-protection-test-july-november-2018/
      We see the same ratings as before, only not in order of ability so MS does look very good, except for the high false positives.
      When ordered by effectiveness we have a relative lineup where I pay attention to the top 5.
      Scroll down and you will see a section that shows the false positive results. You will know you hit the MS section as it is all red.
      It is worth looking at the breakdown as they categorise them into how likely different types of users may come across it.

      Thanks again for your own reply. Your obvious level of competence is why my replies are so thorough (OTT).

  4. There is a way to check HTTPS without MITM – simply halt the browser and check what it has deciphered in parallel and _then_ let it render, or execute (yes the browser executes software too – see spectre or rowhammer and others of that ilk 😉 ) or download if it is clean. The AV is in the system, you can inject its DLL anywhere it wants.

    (It is a bit more complicated than that, there is are several more steps involved, but that is basically the method which works without breaking the encryption of, or the certificates for on the transport. Mind: HTTPS is only a (overrated) transport encryption, it is gone as soon as it is _in_ the browser)

Leave a Reply


This site uses Akismet to reduce spam. Learn how your comment data is processed.