How to Run Compatibility Test Suite (CTS) on Android

Here is a how-to article that explains each and every stage in this important step of getting your Android license.

There is no doubt in anyone’s mind that Android is the most popular OS today for handheld devices, commanding 80% of mobile OS marketshare, and making increased forays in non-mobile embedded devices as well, for instance, in PowerPC for avionics systems.

As a general rule, embedded devices should support a high degree of customization (considering the unpredictable field environments which they will be exposed to). They should be always-on, and present themselves as an always-present gateway to the Internet.

Accordingly, each and every embedded device company desires to customize their devices by extending the functionality of the OS as much as possible. There is a reason why Android was designed as a robust platform for running aftermarket applications.

After customizing Android as per device needs, if the user wants to run Google play store applications, they need to provide what is known as compatibility to these applications.

For that, after customization, the developer needs to test Android compatibility.

For Android compatibility testing, Google provides Compatibility Test Suite (CTS). The CTS is a free, commercial-grade test suite which represents the “mechanism” of compatibility all the way up to Android 7.1.

Google also provides the Compatibility Test Suite Verifier (CTS Verifier) which is a supplement to CTS. CTS Verifier provides tests for APIs and functions that cannot be tested on a stationary device without manual inputs.

For building Android compatibility devices, all CTS tests need to pass. Use the CTS as an ongoing aid to evaluate compatibility during the development process.

We had briefly explained CTS in a previous blog.

CTS

The CTS runs on a desktop machine (Windows or Ubuntu) and executes test cases directly on attached devices or on an emulator.

The CTS contains a set of unit tests designed to be integrated into the daily workflow (such as via a continuous build system) of the engineers building a device.

It provides essential help in revealing Android incompatibilities early on, and ensures that the software remains compatible throughout the development process.

The CTS is an automated testing mechanism which includes two major software components:

• The CTS runs on your desktop or laptop and manages test execution.
• Individual test cases are executed on the Device Under Test (DUT). The test cases are written as JUnit tests in Java and packaged as Android .apk files to run on the actual Android device target.

CTS Verifier

CTS Verifier is used for manual testing which provides tests for APIs and functions that cannot be tested on a stationary device without manual input. Examples include Accelerometer, audio quality etc.

The CTS Verifier is a tool used for manual testing and includes the following software components:

• The CTS verifier application that is executed on the device under test and collects the results.
• The executable(s) or script(s) that are executed on the desktop or laptop to provide data or additional control for some test cases in the CTS Verifier app.

Workflow of CTS

The above figure summarizes CTS workflow with eInfochips’ Eragon 820 (based on Qualcomm Snapdragon 820 processor).

Types of test cases included in CTS

• Unit Tests: To test atomic units of code within the Android platform; e.g. a single class, such as java.util.HashMap.
• Functional Tests: To test a combination of APIs together in a higher-level use-case.
• Robustness Tests: To test the durability of the system under stress.
• Performance Tests: To test the performance of the system against defined benchmarks, for example, rendering the number of frames per second.

Areas covered by CTS Test cases

Test cases cover the following areas to ensure compatibility:

How eInfochips can help

Achieving CTS compatibility for an Android device is a crucial step to get GMS License, and eInfochips just happens to be one of the very few service companies that can help you get GMS License. If a customer wants to validate their product for Android compatibility, we can help in the following ways:

• Test the Android based product and provide detailed analysis report based on result generated by CTS.
• Provide solution for failed CTS test on your product.
• CTS customization targeted to your product and supported functionalities.