McAfee Research Blog

Double Strike by AMTSO

It was very encouraging to see that more than 40 people came to Budapest, Hungary, to discuss and agree on new industry standards as part of the effort undertaken by the Anti-Malware Standards Organization (www.amtso.org.) The awesome historic surroundings set the mood for our discussions.

Seeing such a great turnout in the current economic climate shows how much AMTSO members care about raising the standards of testing anti-malware products. Especially considering the recent rise in the number of rogue security products (such as the now infamous “Anti-virus XP 2009″), it is clear that we need transparent and fair testing more than ever.

AMTSO members finalized and adopted several new documents to the current portfolio. (Have a look at the collection of documents here: www.amtso.org/documents.html.)

But I would like to draw your attention to two papers that, in my opinion, represent very significant steps for the security industry as a whole.

• The first one is “AMTSO Analysis of Reviews Process,” and it presents the process of analyzing reviews. The creation of such a process paves the way to highlight great reviews and/or to expose substandard tests in public. (AMTSO promises to publish all the analyses they undertake.) I really hope that this process, designed to be transparent and fair, will improve the quality of testing and benefit both the developers and consumers of anti-malware technology. If you have doubts that this process is going to be unbiased I will remind you that AMTSO members work for competing security companies, and there would not be a snowball’s chance in hell to agree on the process if it were not designed to be fair. The next step is to put the “AMTSO Analysis of Reviews Process” into practice. I cannot wait to see how it will go.
• “AMTSO Best Practices for Testing In-the-Cloud Security Products” is the second very important milestone. Some anti-virus products started using “cloud” technologies (such as McAfee’s Artemis, which was launched in the beginning of 2008) and the number of cloud-based products is growing; so there is a need to address the fundamental problems associated with testing solutions that are not under the control of the tester. (That is, part of the product is not “in the hands” of the tester; moreover, it can change at any moment in time.) I think it is amazing that representatives of so many competing security companies agreed on fair and scientific principles of how to test cloud-based products. To be honest, when we started this effort we were rather sceptical about finding a sensible way to address all the problems that testers face when evaluating such technologies. The adoption of AMTSO best practices for testing in-the-cloud products means that our brainstorming was successful. I am very pleased to see the agreed results adopted and published. Thanks for that effort go to all the security researchers who contributed to the document and all AMTSO members who voted for it.

Back to McAfee Research Blog overview