# Test Reports

* [Reports](#reports)
* [Running Tests](#running-tests)
  * [Details of Running Tests](#details-of-running-tests)
  * [Actions](#actions)
    * [Live Preview](#live-preview)
    * [Stop Execution](#stop-execution)
    * [Error in Test](#error-in-test)
* [Test Results](#test-results)
  * [Overview of Test Results](#overview-of-test-results)
  * [Icons](#icons)
    * [Test Configuration](#test-configuration)
    * [Scheduled Test Type](#scheduled-test-type)
    * [Plugin Type](#plugin-type)
  * [Actions](#test-result-actions)
    * [View Script/ View Suite](#view-script-view-suite)
    * [Play Video](#play-video)
    * [Email](#email)
    * [Download Report](#download-report)
    * [Retry](#retry)
    * [Create Jira Ticket](#create-ticket)
    * [Delete Report](#delete-report)
  * [Detailed Execution Report](#detailed-execution-report)
    * [Metadata](#metadata)
      * [Test Case Scenarios](#test-case-scenarios)
      * [Actions](#report-actions)
      * [Environment Name](#environment-name)
      * [Date and Time](#date-and-time)
      * [Device Info](#device-info)
    * [Test Report](#test-report)
      * [Test Report Details](#test-report-details)
    * [Performance](#performance)

## Reports

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-a9131be2b470cb6cb392d9571b91fb227b93048e%2FReports-1.png?alt=media)

The reports are located in the top right corner of the screen. Clicking the button will take you to the reports splash page.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-8f9848a9799ab67b1e9f2fe97d38e7e391a3586c%2FReports-2.png?alt=media)

The splash page shows the user the current metrics for tests ran.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-6ab96d9bfd5c809d8c95a9c2df1ef4e6b59ea8b3%2FReports-3.png?alt=media)

Project summary lists all of the test suites test scripts and top users in a project.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-47a0d4833c842451fd10c90d6bed914498e6dc22%2FReports-4.png?alt=media)

The range of metrics can be changed with the graph time selection.

## Running Tests

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-c00cb65d91b3dfdf2d34e678d780c91fb3c64dac%2FReports-12.png?alt=media)

All of the executions that are currently running will be displayed in the “Running Tests” tab. There is a legend on the right most section of the screen that displays how many tests are currently running and how many have encountered an error in starting an execution.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-56f29eefe3a87da1d9b1f6cd1a956173a5fcdd40%2FReports-6.png?alt=media)

Click on the “Refresh” button to manually fetch the latest status of any running tests. Or set the dropdown of “Refresh Time” to automatically update the execution statuses based on the selected time.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-3a69a89a68be37a18e072dc51805a8c14b23e503%2FReports-7.png?alt=media)

The flow of execution status is below:

![Execution\_Status](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-a25193f73e663fcf1e4e2ac605227c769c934d4c%2FExecution_Status.png?alt=media)

The **Test Queued** status will have one of the following information after it:

* **Up Next** - this should be the next test that is executed
* **A number** - how many executions are ahead of the test in the device queue
* **NA** - queue information for the test is not availalbe at the moment

![Test\_Queued](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-c00cb65d91b3dfdf2d34e678d780c91fb3c64dac%2FReports-12.png?alt=media)

### Details of Running Tests

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-bdd1f1363d6d1391d1769297fa3895561efdc5e2%2FReports-9.png?alt=media)

* **App Name:** app file name, app activity or bundle id used to run the execution
* **Test Script Name:** name of script/suite that is being executed
* **Device:** name of selected device used for execution
* **Device Type:** displays whether the selected device is shared or dedicated.
* **Version:** selected device version
* **Start Date Time:** execution start time
* **Status:** current state of execution
* **Actions:** actions available for an execution

### Actions

#### Live Preview

Click on the “Live Preview” button to see the stream from the device as the test is being executed along with the description of each executed step and its Pass/Fail status.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-e0127bc5a0d843b9f9b6d9b9c26112b03ba82545%2FLive%20Preview.png?alt=media)

Once the execution is finished in the Live Preview page, the screen will be routed to the “Test Results” tab.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-17950bd2ce9f0f98523837ef82cd44fe1f8ab3d6%2FLive%20Preview-2.png?alt=media)

#### Stop Execution

Click on the “Stop” button to open the modal below and click the “Stop Run” button to terminate an execution or click the “Cancel” button to continue the test.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-868d6a94d5922e53b2bbe4dd9c8660129a5745da%2FReports-10.png?alt=media)

#### Error in Test

An execution can have this status if one of the following happened:

* Problem with creating a session
  * Previous status was **Allocating Device**
* Problem with updating results
  * Previous status was **Uploading Results**
* Error with queueing test to device
  * Previous status was **Queueing in Progress**

Hover over the “Error in Test” status to learn what went wrong.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-4cf821340f773b81ac880a0dd2a870ccf2dc3b9d%2FReports-11.png?alt=media)

Click on the “Retry” button to try the execution again.

## Test Results

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-c9d05e8ec7d18b08f246fc1fcf70bc92135a920b%2FTestResults-1.png?alt=media)

Displays all the completed tests from the last 6 months and an overview of each execution.

### Overview of Test Results

* **Icon:** Depending on the configuration used to run the execution, there might be an icon to signify that configuration before the app name column of each execution. There are 3 categories of icons:
  * Test Configuration
  * Scheduled Test Type
  * Plugin Type
* **App Name:** app file name, app activity or bundle id used to run the execution
* **Test Name:** name of script/suite that is being executed
* **Start Date Time:** execution start time
* **Duration:** how long an execution took
* **Device:** name of selected device used for execution
* **Device Type:** displays whether the selected device is shared or dedicated.
* **Status:** displays result of an execution
  * Pass
  * Fail
  * Cancelled
* **Actions:** actions available for an execution

### Icons

#### Test Configuration

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-9e8ceba47fa13d29fac80d40d45248972bacb4fe%2FHealerIcon.png?alt=media)

Healer icon will appear if an execution was run with healer option enabled

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-34777c8ca425a4070e6e88c5472d32fffec48df0%2FEmailIcon.png?alt=media)

Email icon will appear if option to email report was selected for execution

#### Scheduled Test Type

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-6567c41ed64e70cc5582a822252eeec8f8c94a7d%2FScheduledOnceIcon.png?alt=media)

Execution was scheduled only once.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-22aa6ed0feb7c3117cafd5909760153c928ba99e%2FScheduledDailyIcon.png?alt=media)

Execution was scheduled daily.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-15da313d1267737466db27fd811e98817289d77c%2FScheduledWeeklyIcon.png?alt=media)

Execution was scheduled on a weekly basis.

#### Plugin Type

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-7c713071b95a0804b5cbe2f462cd81a63baa84b8%2FAzureIcon.png?alt=media)

Execution was triggered from a plugin like Jenkins, Azure or Qyrus CLI

### Test Result Actions

Click on the ellipsis in the “Actions” column to get list of available actions for a test.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-3e6a81b6f476aa0aa2a7dac951177aef8efde4e3%2FTestResults-2.png?alt=media)

#### View Script/ View Suite

If the execution type was script, then the “View Script” option will appear and clicking it will route to the steps page of that script. However, if it was a suite execution, “View Suite” option would be present and clicking it will route to the scripts page of that suite.

#### Play Video

Plays the video recording of the execution.

#### Email

Sends the test report to the emails specified in the modal that opens.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-35a3487f5dc47cfa5b82c44c9cf719aea9ff91cf%2FEmailModal.png?alt=media)

#### Download Report

Downloads the zip report of the test.

#### Retry

Runs the execution again

#### Create Jira Ticket

Creates a Jira ticket using the any linked Jira configurations for the project with the report as an attachment.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-c379ebfa7cd72e79564c199c6a3802f12393f4d5%2FJiraTicketModal.png?alt=media)

#### Delete Report

Opens a modal to delete a report or cancel the deletion.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-203b9e907d753fe04c644378d9c9a8b198134828%2FDeleteReportModal.png?alt=media)

Click on any row of the reports table to view more details about the execution.

### Detailed Execution Report

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-dd9411ed1edb87c236d3c393ae9bfb0611473563%2FDetailedReport-1.png?alt=media)

The page displays the complete report of an execution. This page can be broken down into 3 main sections:

* Metadata
* Test Report
* Performance

#### Metadata

**Test Case Scenarios**

Displays the total number of scripts executed and breaks down how many passed/failed.

**Report Actions**

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-8ebf7790d9dd828bff0da62b3bdb44a6b218f6f8%2FReportVideoIcon.png?alt=media)

Plays recording of execution

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-72544e73b906d999db2d3d0b7c1751a836fbfde0%2FReportDownloadIcon.png?alt=media)

Downloads a zip report of execution

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-2d880f2dfc5aaceb52fef61f22b5a34daa941dc5%2FEmailReportIcon.png?alt=media)

Sends execution report as an email

**Environment Name**

Displays the environment used for global variables during execution

**Date and Time**

Start and end times of execution along with how long it took to complete

**Device Info**

Details about device used for execution

* Device Name
* Device Type (Android/iOS)
* Version

#### Test Report

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-2bc8061fd5922dbb3591ef6883be0cc3ad3030df%2FTestReport.png?alt=media)

The report is broken down by each script in the execution and the name of the script is displayed in the “Test Scenario” column.

**Test Report Details**

* **Test Scenario:** name of script in the execution
* **Steps:** step number
* **Step Description:** description of step. Might include the error message for failed steps
* **Expected Result:** expected result for a verifier action step
* **Actual Result:** actual result of a step. Might be only available for verifier and “Create Variable” actions
* **Device Type:** displays whether the selected device is shared or dedicated.
* **Status:** displays result of a step
  * Pass
  * Fail
* **Execution Time:** displays how long a script took to execute
* **View Screenshot:** clicking on the button opens a screenshot of the device after the step was executed
* **View Script:** clicking on the button routes to the steps page of that script. This option is currently only available for script executions.

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-984388f4b21aca4672ea4ddd08f7b0d68cbf2ca9%2FTestReport-2.png?alt=media)

#### Performance

Displays the following device metrics during the execution:

* Battery Statistics
  * Battery Status i.e. charging
  * Battery Charge
  * Power consumed by app

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-dd8cb53b7ba65cc3f4da9a696cf514137ed5127c%2FBatteryStats.png?alt=media)

* CPU Utilization

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-a95d25c4ccde749862a5993bc75de63daa2f89f0%2FCPUStats.png?alt=media)

* Memory Consumption

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-ea331b11f70397effce3ee5ef153530916927895%2FMemoryStats.png?alt=media)

* Network Usage Statistics

![](https://3407579843-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHgmf0zEqt232nyShGOTi%2Fuploads%2Fgit-blob-229f10c5cd431c96da79b7f0e47f82f9aae22ddb%2FNetworkStats.png?alt=media)

Hover over any of the keys in the legends of the graphs to view only the graph for that key.

\*\*\* **Note:** **This feature is in beta so it might contain some incorrect data. Also, this feature is currently only available for Android devices.** \*\*\*