Visual Test Report

• View a Report

• Pixel Pass

• Similarity Score

• Comparing Baseline and Current Executions

• Skip Regression on a Step

• Navigation

• Filtering

View a Report

Below are the required steps necessesary for a user to view a Visual Test report

Navigate to Reports Page. Select a report with the Visual Testing icon.

visual_testing_icon

Click the Visual Testing Report tab.

visual_testing_reports_page

Once these steps are followed, the Visual Testing report will be displayed and it will look similar to the image below

visual_testing_report_complete

Pixel Pass

Pixel Detection Pass will be located at the bottom of the test step card when in Grid View. Pixel Detection Pass does a pixel by pixel comparison of your current test step with the baseline test step and calculates the percentage of matching pixels between the two screens.

visual_testing_report_pixel_pass

A pass or fail is determined by the Pixel Pass Threshold that is set by the user before execution. The user can set the threshold to any number from 0-100.

Ex. If the user sets the threshold to 95% the Pixel Detection Pass score is 95% or greater, the step will pass. If the score is less than 94%, it is a fail

visual_testing_report_pixel_threshold

Similarity Score

Similarity Score will be located at the bottom of the test step card when in Grid View. Similarity Score uses our AI tool to calculate the similarity in features from current step and the baseline execution. There will be a status attached to it based on it's score. There are 4 different statuses:

• Perfect Match (100% Regression Match)

• Pass (99% - 94% Regression Match)

• Caution (93% - 87% Regression Match)

• Fail (<87% Regression Match)

visual_testing_reports_similarity_score

Comparing Baseline and Current Executions

To view the comparison between the baseline execution and current execution, click on any of the step cards. This will bring the user to Comparison View. To view the Pixel Pass or Similaity Score of the current execution, hover over the status icons in the top right corner.

visual_testing_comparison_view_score

Click on the eye icon to highlight changes between the baseline and current executions. If a Similarity Score is perfect, an eye icon will not appear

visual_testing_after_regression

Perfect and Passing comparison scores will show zero to very insignficant changes. These minor changes may not always show changes within the Comparison View when clicking on the eye icon.

Skip Regression on a Step

If the user does not want to run a regression test on a specific step, the user can disable screenshots for that step before executing the test with Visual Testing. Doing this will result in the step not showing any Pixel Pass or Similarity, as seen below

Note: To execute a Visual Regression test with disabled screenshots, the user must first execute a baseline test with those selected screenshots disabled.

The user can also use the Visual Testing action type if they do not wish to get Visual Testing reports on each step. With the Visual Testing action type, the user can select which steps they would like to get Visual Testing reports on.

Navigation

Click the arrows to navigate through steps. Users can also input a test step number to immediately jump to that test step.

Click any of the step cards on the side panel to navigate to that selected step

The user can click the grid icon to return to Visual Testing Grid view.

Filtering

There are two ways to filter steps:

• Search field

  • filter by step number or description.

• Filtered Screenshots

  • filter by Pixel Pass or Similarity Score

Additional Information

For information on executing a Visual Test, please click here.

For a complete overview on the Visual Testing feature, please click here.