Instrumentation Feature
What is Instrumentation?
Instrumentation is the ability to monitor, measure, or analyze the specific components within a system or application through the use of injected code instructions.
Instrumentation has been implemented before on Qyrus, with the Biometrics feature available to users on the Device Farm Service of the platform. Qyrus is now introducing this feature to their Mobile Testing Service to be used for both Live Test Sessions and Test Executions.
Biometric Instrumentation on Qyrus allows for the monitoring of components within the specified application that utilize or call the libraries responsible for Biometric Authentication. By monitoring these components, the injected code instructions will grant users the ability to bypass this authentication during their Live Test Sessions and Test Executions.
Instrumentation Use Cases
Biometric Instrumentation has considerable use cases when it comes to testing mobile applications. For example, many mobile applications have the ability to opt for using the biometrics information registred on the device as the method of signing in to or accessing the application.
Authentication Flows
Mobile applications such as banking or financial applications have security and authentication as a cornerstone of their user experience and privacy. Authentication flows utilizing biometric authentication provide a greater sense of security for users.
Biometric Authentication flows can now be tested on mobile applications through the use of the Instrumentation Feature now available on the Qyrus Mobile Testing Service. By Enabling Instrumentation for the Live Test Session or Test Execution, users can now test the efficency of their Authentication flows and incorporate use cases such as two-factor authentication that utilizes both Biometric and Simple Authentication.
Instrumentation Security Guidelines
The Qyrus Team has provided additional security guidelines when using the Instrumentation Feature that aid in providing protection and privacy for all users of the platform. The security guidelines are outlined as follows:
Private Instance Usage
Due to the unique nature of the Instrumentation feature, this will be available only by request on private instances and on a per-device basis.
By default, this feature will be disabled on all devices.
App Removal After Test Executions
Test Executions with Instrumentation enabled, require that the target application be installed at the beginning of every test and will automatically be removed from the device after every test.
This will ensure that the debuggable application will not remain on the device.
Access Limitations
The Qyrus Team highly suggests limiting the number of users in a project that utilize the Instrumentation feature, as this will limit the number of users who have access to the debuggable application and minimize security risks.
Instrumentation Restrictions
The current implementation of Biometric Instrumentation on the Qyrus Mobility Testing Service, has certain limitations when it comes to Live Test Sessions and Test Executions.
Instrumentation Prerequisites
The following requirements must first be met for the Instrumentation feature to be enabled on the Qyrus Mobile Testing platform.
1. Debuggable Application
The Instrumentation feature will only work on non-production level applications. For Android Applications, this is done by building the application with the “debuggable” attribute set to “true”.
2. Fingerprint and Passcode Registration
A fingerprint must be registered on the device. The ability to lock and unlock the device by passcode must also be enabled. Fingerprint and passcode registration will be handled by the Qyrus team.
In general, devices are not passcode locked in the Qyrus Device Farm to prevent users from attempting to execute a test script on a locked device. A passcode will be provided to users in the event that this situation happens.
3. Device Registration for Instrumentation
Only devices that are registered for Instrumentation can access this feature. Device registration for Instrumentation is handled by the Qyrus Team.
Execution Limitations
The Instrumentation Feature is only available for Single, Non-Parameterized Script Executions.
Additionally, the Instrumentation Feature is only available for "Install App" Executions and the application file (APK file) specified must be uploaded to Qyrus.
The Instrumentation Feature is unavailable for the following Execution Types:
Test Suite Executions
Multi Test Script Executions
Parameterized Test Script Executions
"Launch App" Executions
Device Limitations
The Instrumentation Feature is currently only available for the following device conditions:
Must be Android Device
Must be Single Device ("Device Coverage" Execution Strategy Option) or Device Pool containing only one device
Device must have biometrics enabled/registered
Must be Dedicated Device
The Instrumentation Feature is currently not available for the following device conditions:
iOS Devices
"First Available" Execution Strategy Option
Device Pools containing more than one device
Devices that do not have biometrics enabled/registered
Shared Devices
A summary of the device and execution limitations for the Instrumentation Feature can also be found when hovering over the info icon of the "Enable Instrumentation" section for both Live Test Sessions and Test Executions.
Action Type Limitations
With the current implementation of the Instrumentation Feature on Mobile Testing being closely tied to the running application being tested, certain action types that close, reset, or terminate the application can interrupt or close the Instrumentation Feature running on the device.
The Instrumentation Feature can be interrupted, blocked, or closed unexpectedly when any of the following action types are used within the Test Script:
Launch App
Switch App
Close App
Terminate App
Reset App
Uninstall App
Enabling Instrumentation
The Instrumentation Feature for Mobile Testing can be enabled for both Live Test Sessions and Test Executions.
Enabling Instrumentation will allow for biometric bypassing on the specified application and will reset and reinstall the application by default.
When enabling Instrumentation, users may also experience longer wait times than usual when connecting to their Live Test Session or initializing their Test Executions, relative to the size of the application choosen.
Instrumentation Feature Option for Test Execution
Enabling Instrumentation for Live Test Sessions
To enable Instrumentation for Live Test Sessions, the following conditions must be met:
The selected Test Script must not be parameterized
The "Install App" option must be selected
The selected device must meet the requirements described in Device Limitations.
In all other cases, the "Enable Instrumentation" option will be disabled for Live Test Sessions.
Once these conditions have been met the "Enable Instrumentation" option can be found after exapnding the Advanced Options menu of the setup modal for the Live Test Session.
Instrumentation Feature Option for Live Test Session
Enabling Instrumentation for Test Executions
To enable Instrumentation for Test Executions, the following conditions must be met:
The Execution Type must meet the requirements stated in Execution Limitations
The selected device must meet the requirements stated in Device Limitations
In all other cases, the "Enable Instrumentation" option will be disabled for Test Executions.
Additionally, it is also strongly advised for the selected Test Script to not contain any of the Action Types specfied in Action Type Limitations for optimal Instrumentation performance.
Once these conditions have been met the "Enable Instrumentation" option can be found underneath the "Reset App" option. Hovering over the info icon will also give a summary of the Limitations for Instrumentation.
Instrumentation Feature Option for Test Execution
Enabling Instrumentation for Dry Runs
To enable Instrumentation for Dry Runs, the following conditions must be met:
The Execution Type must meet the requirements stated in Execution Limitations
The selected device must meet the requirements stated in Device Limitations
In all other cases, the "Enable Instrumentation" option will be disabled for Dry Runs.
Additionally, it is also strongly advised for the selected Test Script to not contain any of the Action Types specfied in Action Type Limitations for optimal Instrumentation performance.
Once these conditions have been met the "Enable Instrumentation" option can be found underneath the "Reset App" option. Hovering over the info icon will also give a summary of the Limitations for Instrumentation.
Instrumentation Feature Option for Dry Run
Last updated
