It shouldn’t strike anyone as a surprise to learn that sometimes, companies cheat. It could be performance results or an exaggeration of its features. The sad thing is, from a business standpoint, it might make sense to cheat. After all, lest the truth be revealed, that particular product might not sell so well. That hurts their own pockets, and sometimes, their shareholders’ pockets. Great for them, horrible for misled consumers.
It could be considered sad that despite a countless number of companies being found out for cheating before, and ultimately result in being publicly humiliated, they still continue to cheat… in hopes that they’ll just get away with it. Sometimes, this happens on a minor level; other times it’s downright major, as we’re seeing right now with Volkswagen.
When news first broke of Volkswagen’s emissions debacle, I couldn’t help but draw similarities to a similar debacle that happened about a decade ago. Some GPU vendors were caught optimizing their drivers in order to achieve higher-than-expected 3DMark scores. In effect, the resulting 3DMark score wasn’t actually representative of regular game scaling, which ultimately meant that consumers were misled. VW pulled off a similar stunt: whenever official tests were detected, TDI-equipped vehicles automatically changed various perimeters in order to deliver improved test results. Unlike the mishap of those GPU vendors, VW’s poor decision can actually cost lives.
Nonetheless, Futuremark has drawn the same comparison, and has decided to reiterate its stance on cheaters. Ultimately, the company has no patience for those who try to fool the system, or consumers. It mentions that unlike many other benchmarking services, it doesn’t allow prerelease hardware (this includes mobile devices) to “leak”, because oftentimes, those can be deliberately misleading as well.
The company points to official guidelines that hardware manufacturers must follow, and notes that only results acquired through the use of WHQL drivers are ever accepted as official.
Nearly two-years-ago, we saw Futuremark delist devices from its charts that were deemed to be the result of cheating, so as with many things, even detecting the cheaters is a process that must evolve over time.
In that regard, Futuremark sure has its job cut out for it.