Mobile SoC Maker MediaTek Caught Cheating In Benchmarks, Results In UL Delisting

SoC maker MediaTek has been in the news quite a bit this week, and not for the greatest of reasons. Cheating in benchmarks isn’t new, but it’s still railed against by anyone who cares about accurate data. In the mobile world, we’ve seen some companies optimize for benchmarks in the past, but usually, they are quickly detected, and delistings help teach the offender a lesson fairly quickly.

Enter MediaTek. Last week, AnandTech wrote a piece on oddness with the performance results seen from the Oppo Reno3 Pro, which bundled in a European version of MediaTek’s Helio P95 SoC. Later, the publication received a Chinese version of the Reno3, which utilizes the MediaTek Dimensity 1000L SoC. Despite being the more technically powerful of the two, this latter SoC performed worse than the original.

Oppo’s Reno3 Pro

After the hint of cheating, UL sent AT an anonymized version of PCMark, one that would escape the sensors a vendor like MediaTek would have in place. Lo and behold, the performance from this mystery PCMark build delivered vastly different results – we’re talking a drop from a 9,048 work score, down to 6,781. That’s not exactly insignificant.

As it turns out, there’s a “power_whitelist_cfg.xml” file on some MediaTek-powered phones that sits inside of the /vendor/etc folder. Inside will be an XML file storing profiles of common benchmarks, including most of the ones you’d be familiar with. It even includes some AI benchmarks that haven’t become common until fairly recently.

After calling MediaTek out, AnandTech received a response that effectively ignored the actual problem, and since then, the company has only doubled-down. Well, today, the folks at UL have made it known that these kinds of shenanigans are in fact a problem, because the company has delisted all MediaTek-powered devices that feature any one of these chipsets: G90, G70, P95, P90, P65, P60, P20, and A22.

Before and after PCMark benchmarks; Credit: AnandTech

UL notes that this delisting is temporary, as it usually is. Any vendor caught cheating can rectify their actions, and in effect work towards being trustworthy again. With MediaTek confident it’s done nothing wrong, however, we only wonder if the company will try harder with these tricks down-the-road. The upside is that people will be watching closely, going forward.

UL doesn’t make decisions like these on a whim, so the fact that it’s taken action means that MediaTek has unequivocally cheated, something the industry at large, including our journalist friends, seem to agree on.

At this point, all we can do is see if MediaTek changes its tune. Otherwise, consumers suddenly have to take benchmark results from these affected devices with a grain of salt.