Technology analysts are debating a research company’s claims that Intel Corp.’s (NASDAQ:INTC) smartphone application processor outperformed its ARM-based rivals like Nvidia (NASDAQ:NVDA), Qualcomm (NASDAQ:QCOM), and Samsung, in a recent benchmarking exercise.
Allied Business Intelligence Inc., which trades as ABI Research, released a press release regarding its results after conducting benchmark testing on a number of smartphone application processors. ABI Research concluded that Intel Corp. had succeeded in significantly reducing the power consumption of its smartphone application processor and that it now rivalled and is often lower than equivalents based on the ARM architecture licensed from ARM Holdings PLC (NASDAQ:ARMH).
According to the research company, the benchmarks were conducted by comparing the performance of the Lenovo K900 smartphone, powered by Intel’s Z2580 application processor and supporting XMM6360 chipset, with a series of Samsung smartphones that are based on ARM-based application processors from Nvidia, Qualcomm, and Samsung.
Vice president of engineering at ABI Research, Jim Mielke, said the benchmarks were impressive. “But the real surprise was the current consumption recorded during the benchmarks,” he said. “The new processor not only outperformed the competition in performance but it did so with up to half the current drain.
“Intel did significant work to bring the current drain down on their well-recognized high-performance processors but the competitors did not help themselves” Mielke continued. “The ARM architecture used by nearly all of Intel’s competitors is well known for its low power performance but in bringing the processing power up closer to PC levels, the current drain has taken a significant hit.
“Combining the high-end modems — the XMM6360 is used in both the Lenovo K900 and the Samsung Galaxy S4 i9500 — with their application processors for high- to mid-tier solutions and single-chip EDGE chips for low-cost phones makes Intel a rare full portfolio provider,” he said. However, many technology analysts disagree with Mielke’s findings and are questioning the benchmark standards used to justify ABI Research’s conclusion.
EE Times analyst Jim McGregor said ABI Research’s blanket statement proclaims Intel has surpassed the entire ARM ecosystem in mobile processors for the all-important high-end smartphone segment. McGregor suggests ABI Research used the AnTuTu benchmark method, as several analysts have confirmed Intel processors perform better under its tests.
“Benchmarks have always been subject to manipulation,” McGregor said. ”It has always been in the best interest of the technology vendors to demonstrate their goods in the most positive light possible. As a result, vendors have attempted to manipulate benchmarks through various means, ranging from optimizing hardware configurations or modifying software to match the benchmark testing parameters to even attempting to influence the benchmark code or methodology.
“As a result, no benchmark is completely accurate in evaluating a processor or device,” he added. “However, where one benchmark falls short, another typically excels. This is why is has become more common practice to use a suite of benchmarks in product evaluations. So, the first thing that strikes me as odd is that the only benchmark referred to in the articles relating to the new Intel processor outperforming all the ARM competitors is AnTuTu.”
Although ABI Research did not specify which benchmarks were used in its press release, McGregor said Intel and other tech reviewers confirmed that they experienced similar positive results using the AnTuTu benchmark method. To illustrate his concerns about the AnTuTu benchmark, McGregor compiled a variety of benchmark information from tech reviewers, benchmarking organizations, and other industry resources.
The compiled results (see chart below) show the Intel chip not faring as well in other benchmarks. The results demonstrate a significant advantage for the Intel processor relative to the Samsung processor but only in the AnTuTu benchmark, and only in one of the AnTuTu benchmark tests.
