Last week, security research firm Accuvant released a report – commissioned by Google – which claimed that Chrome was the most ‘secure’ browser and Firefox was the least secure. Now, rival security firm NSS Labs has published a report claiming that Accuvant’s methodology is flawed, and is part of a “larger strategic move by Google to eliminate the competition”.
The report, entitled “The Browser Wars Just Got Ugly”, raises a number of questions about the investigative methodology employed by Accuvant during the creation of the report. NSS Labs allege that “the clear bias in the test methodology” diminishes the report, from what is otherwise a respected research institution, into a propaganda piece for Google.
“Accuvant disabled highly relevant portions of non-Google browsers’ protection without noting the impact on the overall results,” NSS researchers note in their report. “This error in testing resulted in an erroneously negative assessment of the browsers’ protection capabilities, since some browsers will only block malware during or after download and before execution.
“The JIT hardening analysis failed to give ample credit to the more proactive technologies employed by IE9, which happened to not be present in Chrome,” NSS further alleges in the report.
The authors that wrote the NSS report also question the timing of the Accuvant study’s release, which coincided with the expiring of the Google-Firefox funding deal. This deal, which expired in late November, made up 84% of Mozilla’s $123 million in revenue in the 2010 fiscal year.
In an interview with Computerworld, NSS Chief Technology Officer Vikram Phatak told tech-journalist Gregg Keizer that: “This is a vendor-funded paper, and in these cases, the vendor is going to drive the methodology [of the testing], which appears to be the case here.”
In a footnote in their report, NSS noted that it analyzed Accuvant’s study at the request of some of its customers.