Visual Testing Execution

• Executing a Visual Test

  • Baseline Execution Requirements

    • Baseline Execution

    • Device Requirements

    • Global Settings

      • Ignore Header

      • Ignore Footer

      • Pixel Pass

Visual_Testing_Toggle

Visual Testing allows the user to test their application layout with visual regression.

Executing a Visual Test

Visual Testing can be enabled in the Test Execution menu. Currently, only Visual Regression testing is available.

1. Navigate to the Test Execution Page in mobility.

2. Select/enter app details.

3. Select a device to execute test.

4. Click the “Visual Regression” toggle button.

   • Once enabled, previous executions of the script on the same device from Step 3_ will appear

   • If no baseline executions are found, refer to the baseline requirements here

   • If the “Visual Regression” toggle stays disabled, refer to the conditions below.

5. Select a baseline execution.

6. Click "Execute Test" button.

There will also be accessibility options for visual testing coming soon.

Visual Testing Requirements

• Visual Testing is only available for single, non-parameterized script executions.

Baseline Execution Requirements

Visual_Testing_Tooltip

Baseline Execution

• A baseline is a previous test execution on the current version of the test script on the same device.

• The Global Variable Environment used for the baseline execution can be different than the current Global Variable Environment selected. The Global Variable Environment information will be shown when selecting a baseline execution.

• Modifying a test script will invalidate any previous basline executions. An execution must be run after the test script was last modified to be considered a baseline.

Device Requirements

• Only one device can be selected for Visual Testing.

  • If a device pool is being used, it must only have one device or Visual Testing will be disabled.

  • Using first available device is not an issue as long as the device pool only contains one device. However, if first available device is selected with no device pool, Visual Testing will be disabled.

• The current device must have the same device ID/name as the baseline execution.

If these requirements are met, Visual Testing options will appear like below.

Global Settings

• Global Settings for Visual Test Executions allow for users to fine tune their testing by excluding regions from screenshots and setting a passing rate threshold.

  Ignore Header

  • The Ignore Header global setting will exclude the system bar area of the device screenshot from the visual comparison.

  Ignore Footer

  • The Ignore Footer global setting will exclude the navigation bar area of the device screenshot from the visual comparison.

• The 'Ignore Header' and 'Ignore Footer' global settings aid in improving the visual testing comparison results by narrowing the comparison to the application screen rather than the entire device display.

• Ignoring specific regions of the device screenshot is also available through the use of the 'Verify Screen' action type. For more information on the 'Verify Screen' action type, please click here.

  Pixel Pass

  • The Pixel Pass global setting allows users to set the threshold for a passing score of a visual comparison.

  • The value for this global setting is set to a passing score of 95 by default.

These Visual Testing Execution global settings can be enabled when selecting a baseline execution.

Additional Information

For information on viewing the report of a Visual Test, please click here.

For a complete overview on the Visual Testing feature, please click here.