Wednesday, October 02, 2013

Benchmark optimizations inflate scores for Galaxy Note 3 by 20 percent

Samsung's done it again, but not a good thing. After being caught inflating the benchmark scores of the international Galaxy S4's GPU, the Korean giant and Apple's arch-rival has been caught doing the same with the Galaxy Note 3's benchmark scores. As noted by Ars Technica on Oct. 1, this is the first time Samsung has been caught boost benchmarks on a U.S. device.

The faux scores we noticed when folks discovered that the processor in the Galaxy Note 3, a 2.3GHz Qualcomm Snapdragon 800 system-on-a-chip (SOC) totally blew away the LG G2, which also sports a 2.3GHz Qualcomm Snapdragon 800. The obvious comment to those results would be "Huh?"

The answer was that Samsung appears to be putting the SOC into a special, high-power CPU mode whenever the device runs a large number of popular benchmarking apps.

Here's what the Note 3 does: Whenever an affected benchmarking app is run, even if the device is idling, the CPU is locked at 2.3Ghz, its fastest possible speed, with all four cores running. In a normal idle state, the three of the four Note 3 cores are shut down to conserve power, and the remaining core drops to a power-saving 300MHz speed.

By changing the package name of Geekbench, Ars was able to trick the Note 3 into running normally, not in the special mode. When the benchmark was run that way, the Note 3 was slightly faster than the LG G2, but not ridiculously so, as it had previously been.

The boost appears to be around 20 percent, which is a huge difference between the benchmarks, boosted and non-boosted. There was also some evidence that points to Samsung tweaking the GPU results as well, but that wasn't solidly concluded.

It's a sad state of affairs to see Samsung producing faux performance numbers, and no doubt Apple fanboys will pounce on this.

This is more than simple marketing puffery, as when Apple called its high resolution display Retina. Puffery, although misleading, isn't the same as out-and-out falsifying of results.

No comments: