IntegrationWithTestManagementTools
Integration with Test Management Tools
Overview
The Qyrus platform currently has integration services with the Xray Testrail and Jira management tools. The configurations for these services vary but they all update the user’s selected management tools.
Configuration Setup Xray
First build Xray integration by navigating to Xray integration page, from dashboard, to learn how to integrate Xray in integration page, follow steps below:
Go to Xray integration, by clicking on Xray card (1) as shown in image above.
please fill all the fields, as explained below:
Primary details (1): Configuration Name: This will be the unique name of Xray configuration. Client ID: Enter Xray client ID client Secret: Enter Xray client secret Note: To learn, how to generate client secret in Xray, please refer to: https://docs.getxray.app/display/XRAYCLOUD/Global+Settings%3A+API+Keys
Jira configuration connection:
If user want to create new Jira configuration to connect to Xray, click on
create new jira confirationTo learn about Jira connection with Xray, please refer to the Jira section below.
If user already have Jira configuration, select Jira configuration from the drop-down menu (3)
This will load all of Jira projects, scroll and select all the projects as per requirement.
Now create configuration button (4) will be enabled, user can click and Xray configuration setup is ready, now the user can proceed to Mobility service to connect Mobility project test.
To create new Xray configuration, select
+(1) icon as shown in image below.To edit Xray configuration to update projects or update any other credential, click on edit button (2) shown in image below.
To delete Xray configuration, click on delete button (3) shown in image below.
To update Xray configuration, click on
update configurationbutton (4) as shown in image below.
Once user have connected Xray, the user can navigate to connected mobility project and click the Xray icon on their script. The integration menu is shown below.
Follow following steps to integrate the test:
choose Xray configuration: This dropdown will list all the Xray configuration connected with user current project, please select relevant Xray configuration and proceed.Qyrus Script: This dropdown is non-editable and will provide name of current project.Xray Issues: This Dropdown will list all the Xray issue connected in Xray configuration, please select relevant Xray issue and proceed.execute test: After Xray integration, navigate to test step window. All the steps should be added to the Xray integration in Jira. Select execution configuration according to your test requirement. Executed test can be checked in the Jira tool. An example of the execution on Jira is shown below.
Check completed test run in test run tab as shown above.
Check all the test in result page by clicking on square icon in test runs as shown above.
Verify all the step results in results page.
Configuration Setup Jira
If the user has not created any Jira configuration, go to Jira integration, by clicking on Jira card (2) as shown in overview section, and follow below steps:
To create new Jira configuration, select
+(1) icon as shown in image below.To edit Jira configuration to update projects or update any other credential, click on edit button (2) shown in image below.
To delete Jira configuration, click on delete button (3) shown in image below.
To update Jira configuration, click on
update configurationbutton (4) as shown in image below.
To create Jira configuration, click on new configuration window:please fill all the fields, as explained below: Integrating with Jira tools:
configuration name: This will be the unique name of Xray configuration.
Domain URL: This will be the name of organization application connected with Jira.
user Email: This will be user mail which have authorization from Jira.
API token: And this will be integration api token user can generate using Jira platform by navigating here.
To learn how to integrate Xray in Jira project, please https://docs.getxray.app/display/XRAYCLOUD/Quick+Setup
Integrating with mobile testing:
Teams: Here user can select from list of all the teams they are part of.
Project: Here user can select from list of all the project from selected team and services.
Configuration Setup Testrail
Navigate to the Integrations page either by clicking on the integrations icon
on the side navbar or by clicking on the icon from the dropdown of the person icon.
Click on the TestRail icon to view/create the configurations.
If there are no existing configurations, a form to create a new configuration will appear.
If there are existing configurations, there will be a list of configurations on the left panel and a detailed view of the configuration on the right.
To edit an existing configuration, click on the pencil once the configuration is selected from the left panel.
Click on the plus icon to create a new configuration.
Required fields in the form are marked the red asterisk:
Configuration Name
URL of the TestRail account
Username
Password
Click on “Get TestRail Projects” button to get a list of projects* associated with the account information in the form. Select one project from the list.
Below the TestRail projects, there is a section for selecting Qyrus projects which should display all the teams, services and projects that the current user is part of.
In order to create a configuration, the user must select one TestRail project along with at least one Qyrus project.
***Keep in mind that a Qyrus project can either be linked to TestRail or Xray and not both ***
User can use the status toggle to make the configuration active/inactive. Inactive configurations cannot be used to link test cases from TestRail to Qyrus scripts.
A Configuration can be deleted by clicking on the trash icon and then the delete button in the confirmation dialog box. When a configuration is deleted, all the linked scripts of the configuration are also deleted.
Linking Qyrus Test Suite to TestRail Test Run
Navigate to the project a TestRail configuration is linked to. Hover over a Test Suite to view the TestRail icon over the script. Click on the icon to open a panel to the right.
Choose a configuration and that will trigger a call to fetch the test plans within the TestRail project that is linked to the configuration. Once a toast appears that the test plans are found, select a test plan from dropdown if available.
Choosing a test plan from the dropdown menu will trigger a call to fetch the test runs available within the selected test plan. Selecting "None" as the value of the test plan field will show test runs not associated with a test plan. Test runs are considered "available" if they are not already linked to a Qyrus Test Suite.
Once test run has been selected the user may link their Qyrus Test Suite to their TestRail test run using the "Link" button.
Linking Qyrus Global Variable Environments
to TestRail Test Runs
After selecting the TestRail Configuration, TestRail Test Plan, and TestRail Test Run that you would like to Link to the Test Suite, as disscussed in the previous section, and additional Global Variable Environments exist in the Qyrus Project, the button underneath "Link Additional Environments" will be enabled.
***The TestRail Test Run that you link to the Test Suite will be used for the "Global" Global Variable Environment. The "Global" Global Variable Environment is the Default Global Variable Environment that gets created with each Qyrus Project. When executing a Linked Qyrus Test Suite or Test Script with the "Global" Global Variable Environment, the execution results will be sent to the corresponding Tests within the TestRail Test Run selected in the third drop-down.***
Pressing the button underneath "Link Additional Environments" will show fields for specifying the Qyrus Global Variable Environment and TestRail Test Run that you would like to link.
Once the Qyrus Global Variable Environment and TestRail Test Run have been specified in their respective fields, the user can choose to add another link to an additional Global Variable Environment.
***Please note that one Qyrus Global Variable Environment can only be linked to one TestRail Test Run. However, one TestRail Test Run has the ability to be linked to multiple Qyrus Global Variable Environments. Additionally, Qyrus Global Variable Environments can only be linked to TestRail Test Runs pertaining to the TestRail Test Plan that the Qyrus Test Suite is linked to (specified in second drop-down).***
When the user has specfied all the necessary information for linking their desired Qyrus Global Variable Environments to TestRail Test Runs, the "Link" button can be clicked to confirm this Link relationship between the Qyrus Test Suite and Qyrus Global Variable Environments to TestRail Test Runs.
Linking Qyrus Test Scripts to TestRail Tests
Navigate to the project a TestRail configuration is linked to. Hover over a script to view the TestRail icon over the script. Click on the icon to open a panel to the right.
***Screenshots below are for the Mobile Testing service, but can find the same functionality in Web Testing if you navigate to the Test Lab section of the service***
Choose a configuration and that will trigger a call to fetch the test cases* within the TestRail project that is linked to the configuration. Once a toast appears that the test cases are found, select a test case to link with the Qyrus script.
As long as the test case is not already linked to another Qyrus script, the linking should be successful, and a success toast would be displayed. After a script is linked, it would always be marked by a TestRail icon and suite which the script is in will display the TestRail icon as well.
Click on the TestRail icon on a linked script, to view the configuration that is used for linking as well as the option to unlink the script from the test case in TestRail.
Click on the Unlink button and confirm the unlinking in the dialog box that follows to remove the link between the Qyrus script and the test case in TestRail.
Relinking Qyrus Test Suite
The user is now able to relink their scripts to TestRail as shown below
Once the user clicks the relink button above they will have the option to select their Test Plan and Test Run again from TestRail.
The relinking functionality lets the user create a new link to their Test Plan and Test Run. Test Scripts linked to TestRail within this Test Suite will be linked to tests within the new Test Run that share the same TestRail Case ID. If no tests sharing the same TestRail Case ID are found then, Test Scripts within the Test Suite will be unlinked.
Once the user clicks the Relink button again the message below will display. Once the user clicks confirm the Test Scripts will be relinked.
Relinking Qyrus Global Variable Environments
The user is now able to relink their Global Variable Environments to TestRail as shown below
Once the user clicks the relink button above they will have the option to select the new TestRail Test Plan and TestRail Test Run that they would like to link the Qyrus Test Suite to.
***Changing the Test Plan that the Qyrus Test Suite is linked to will reset the Global Variable Environment Links currently in place.***
The relinking functionality for Qyrus Global Variable Environments lets the user delete or modify existing Global Variable Environment links for the Qyrus Test Suite, while also providing the ability to create new Global Variable Environment Links. Relinking of Test Scripts that are linked within the Test Suite will occur at the time of test execution when executing with a linked Global Variable Environment.
Once the user clicks the Relink button again the message below will display. Once the user clicks confirm the Global Variable Environments will be relinked.
Execution With Linked Qyrus Scripts, Suites, and Global Variable Environments
If a linked script or suite which has linked script(s) to TestRail is executed, the linked TestRail test run will be updated with the execution results of the Qyrus Test Script.
Each TestRail test linked to a Qyrus Test Script will be updated with details of the execution results and will include a Qyrus Report of test results. Each of the test cases would have a comment as well which has more details about the execution like the time the execution was triggered on and the device/ browser the execution ran on.
When executing a Linked Test Suite or Linked Test Script with a Linked Global Variable Environment, the execution results for the linked Test Scripts will be sent to the corresponding tests with the same TestRail Case ID within the TestRail Test Run that the selected Global Variable Environment is linked to. If no tests sharing the same TestRail Case ID are found then the execution result will not be sent to TestRail.
Notes
If a configuration is disabled, it would be unavailable to link scripts. Also, if a configuration has linked scripts already when it is disabled, the linked scripts will not have the TestRail icon over it if there are no other available configurations for the project. If there are other configurations, click on the TestRail icon for linked scripts that are part of the disabled configuration and unlink them in order to link them to other configurations.
If a linked script is part of an execution where the configuration is disabled, that execution will not create a new test run or update the results on TestRail.
Due to the TestRail APIs using pagination, only the first 250 projects will be displayed and only the first 250 test cases within each suite of a project will be available to link to a Qyrus script for the current iteration of the integration.
Last updated