Developer Console
Sign In
Sign In
Developer Console

Test Summary and Results

On the Tests Summary tab, you can monitor and manage the tests that you submitted. Data contained in this tab can help you with analysis and troubleshooting.

A table of test runs and their current status.
Tests Summary tab

Test runs

After you select Tests Summary , you will see a table containing details about each test that you've submitted. The following table provides a description of the details.

Test summary fields
Field Description
Test Name Name of the specific test that was run. You chose this when you created the test run.
Test Type Type of test case that was run. Test type categories are:
  • Performance/Core Functionality
  • Performance
  • Core Functionality
Test Run Status The current status of the test run. Test Run Status categories are:
  • In Progress: Tests that are currently being executed or are queued for execution
  • Complete: Tests that have successfully finished running on the specified devices
  • Error: Tests that encountered issues or failures during execution
OS Indicates the OS the test was run on.
Device Family Devices the test was run on. Device categories are:
  • Fire TV
  • Fire Tablet
  • Non-Amazon device
Submission Date Date and time when the test was started.

To see detailed results for a test run, select the test name.

Test result summary

When you select a test name, the Test Result Summary screen opens. From this screen you can gather more detailed insights about your test run. The results list the names of the devices the test was run on, and indicates whether that device passed or failed. Select the name of the device to open additional details about the test run.

Core functionality tests

With the core functionality test results, you can analyze test outcomes in detail and identify potential issues, which can help you optimize your app across different devices and operating systems.

Core functionality test: shows number of compatible and non-compatible devices. Table of devices with Pass or Fail status.
Core functionality test results

Core functionality test results provide the following details:

  • Number of devices on which the functional test was run and the number of compatible and non-compatible devices. If a device passes, it's counted as compatible, otherwise it's counted as non-compatible.
  • Names of the devices on which the test was run and a status for each. For details on test statuses, see Device test status .

Select a device from the list to access these additional details:

  • Result of the install, launch, and exit tests
  • Device logs

Performance tests

Performance tests generate data on key performance indicators (KPIs), which you can use to analyze potential bottlenecks (factors that limit performance) for your app. You can use the performance test results to help you optimize resource usage across various devices and operating systems.

Performance test: shows average time to display first frame for devices. Table of devices with Pass or Fail status.
Performance test results

Performance test results provide the following details:

  • Summary of the test results for each KPI.
  • Average value of each KPI based on the number of iterations or test runs performed.
  • Names of the devices on which the test was run and a status for each. For details on test statuses, see Device test status .

Select a device from the list to access these additional details:

  • Results for each KPI. This might include charts or graphs that can help you identify performance trends and anomalies.
  • Data on individual iterations or test runs to help you investigate specific instances of performance deviations or issues.

Device test status

The following table describes the possible statuses a for a test run on a device.

Status Description
Pass The test run completed successfully on the device without encountering any issues.
Fail During the test run, one or more test steps or test cases failed to meet the expected criteria or requirements.
Error The test run could not proceed due to unavailable devices or a failure in one of the dependent systems required for the test. This could be caused by a timeout while waiting for resources or an outage in a supporting service.

Last updated: Jul 19, 2024