OnePlus and Meizu have been cheating benchmarks for some reason
Why are manufacturers still cheating on benchmarks?
An XDA Developer investigation in cooperation with our friends at Primate Labs, the company behind the excellent Geekbench test suite for Android, has uncovered a vast conspiracy of benchmarking infractions across thousands of Android devices.
OK, that's not entirely true, but it sounds good, right? In reality, it was discovered that, as Samsung, HTC and others were found to be doing a number of years ago, OnePlus and Meizu activated "performance" modes on their flagships, the OnePlus 3T and Meizu Pro 6 Plus, when entering some CPU and GPU benchmarks. From XDA:
We came to the conclusion that OnePlus must be targeting specific applications. Our hypothesis was that OnePlus was targeting these benchmarks by name, and was entering an alternate CPU scaling mode to pump up their benchmark scores.
The OnePlus 3T was looking for Geekbench, AnTuTu, Androbench, Quadrant, Vellamo, and GFXBench. As by this point we had fairly clear evidence that OnePlus was engaging in benchmark cheating.
XDA Developers worked with Primate Labs to obtain a special build of Geekbench that wouldn't trigger the cheating, and found significant performance differences and, of course, higher overall temperatures as a result. Similar results were found with the Meizu Pro 6 Plus, which isn't popular outside of China.
For its part, OnePlus says that there was no malice intended in the optimizations, and that they were meant to improve overall game performance, not benchmarks, at the expense of a bit more heat output and battery usage. A future build will remove the benchmark optimizations.
Of course, a level playing field is necessary in an industry where benchmarks, for a small number of users, inform buying decisions, but we always recommend looking at the entire device — not just the speed of its cores — as a measure for purchase.
No comments: