EEMBC Brings Order to the Chaos of Android Benchmarking

AndEBench Provides Honest Measurement of Android Smartphones and Other Connected Devices

EL DORADO HILLS, Calif. — March 15, 2012 — The Embedded Microprocessor Benchmark Consortium (EEMBC) today launched its standardized, industry-approved method of evaluating Android-enabled devices. The EEMBC benchmark, named AndEBench, is available for download on Android Market, now part of Google Play, and soon at the Amazon Appstore for Android. While of great value to processor, system, and software vendors, the easy-to-run AndEBench tool also empowers end-users to validate and compare operations on their phones or tablets, many of which vary considerably in performance.

Ultimately, AndEBench will comprise a battery of performance tests for mobile devices, but its initial focus is on the CPU and Dalvik interpreter performance. AndEBench 1.0 compares the Android platform’s native and Java performance. This will give users a quantified measure of the Java interpreters’ efficiency on a given platform and help them understand what performance gains are feasible.

AndEBench also tests a device’s multicore and/or multithreading capabilities. The user can change the number of executable threads to allow a comparison between single and multiple core operation. For example, AndEBench has measured the Kindle Fire’s native performance as 1370 and 2720 iterations per second, respectively, for single and dual core operation; the Java performance is 94 and 146 iterations per second, respectively, for single and dual core operation. Compare these scores to those of the Velocity Micro T301 CRUZ tablet - 470 native and 17 Java.

The working group that produced the AndEBench benchmark was coordinated by Shay Gal-On, EEMBC’s director of technology. “We developed AndEBench with a strong focus on showcasing the under-the-hood device behavior – and although AndEBench is not a fancy benchmark, it analyzes the processor’s capability as well as that of the Dalvik interpreter’s efficiency,” said Mr. Gal-On. “Furthermore, the collaborative effort of our working group members has ensured that AndEBench provides an equitable, unbiased, and repeatable test for mobile devices –critical to deliver data that can be used by technology providers and customers to fairly assess device performance.”

“Other Android benchmarks available on the market are ‘mom-and-pop’ benchmarks in which the motivation of the benchmark developer is unknown and results can vary significantly. Furthermore, since source code is typically unavailable, it is nearly impossible to verify the functionality or true usefulness of these benchmarks,” said Markus Levy, EEMBC’s president. “EEMBC provides an industry-approved benchmark where the source code is available to members and licensees, providing a full understanding of the code internals, to encourage hardware and software developers to use the results to drive product improvements that matter to end-users.”

EEMBC encourages all vendors and manufacturers to join the consortium’s working group to contribute to future AndEBench versions. These versions could include stress testing for features and functions such as OS layer calls, graphics, audio, networking, floating point, and SIMD functions. To join the working group or gain access to the source code for AndEBench, contact Markus Levy for details.

About EEMBC

EEMBC, the Embedded Microprocessor Benchmark Consortium, develops benchmarks to test embedded processors and systems. The embedded processor benchmark algorithms and applications developed by EEMBC are organized into benchmark suites targeting consumer, digital entertainment, networking, automotive/industrial, telecommunications, Java, and office equipment products. An additional suite of benchmarks, called MultiBench, specifically targets the capabilities of multicore processors. EEMBC also provides a tool called EnergyBench that concurrently measures the performance and energy consumption of a processor. The EEMBC system benchmarks represent real-world scenarios that test deep packet inspection performance, browsing platform performance, and Android performance. These benchmarks may be obtained by joining EEMBC's open membership or through a corporate or university licensing program. The CoreMark benchmark is available for free download at www.coremark.org. The EEMBC Technology Center manages development of new benchmark software and certifies benchmark test results.

EEMBC’s members include ACE Associated Compiler Experts, AMD, Analog Devices, Andes Technology, Applied Micro, ARM, Broadcom, Cavium Networks, Cypress Semiconductor, Dell Computer, Freescale Semiconductor, Fujitsu Semiconductor, Green Hills Software, IAR Systems, IBM, Imagination Technologies, Infineon Technologies, Intel, Lockheed Martin, LSI, Marvell Semiconductor, MediaTek, Mentor Graphics, Microchip Technology, MIPS Technologies, MStar Semiconductor, Nokia, Nokia Siemens Networks, NVIDIA, NXP Semiconductors, Qualcomm, Realtek Semiconductor, Red Hat, Renesas Electronics, Research In Motion, Samsung Electronics, Sony Computer Entertainment, STMicroelectronics, Synopsys, Texas Instruments, TOPS Systems, Wind River Systems, and Xilinx.