Create Job
Evaluate jobs perform comprehensive validations on a single data source.
Step 1: Job Type Selection
Select Job Type as Evaluate.
Enter a Description for the job.
Click Next.
Step 2: Connection Configuration
Select the Data Source Connector from the dropdown.
Provide the required Connection Information.
Click Verify to test the connection.
Once verified successfully, click Next.
Step 3: Adding Verifications
Configure data validations for the selected data source.
Supported Verification Types
Row Count Verification – Validates total number of records.
Column Datatype Verification – Ensures columns match expected data types.
Column Duplicates Verification – Identifies duplicate records.
Column Not Null Verification – Checks for null values.
Column Verification using Regex – Validates values using regular expressions.
Column Verification using Python UDFs – Applies custom Python functions.
Data Source Information appears at the top of the screen with column names and data types (available for database connectors).
Configuration Steps
Enable the desired verification option(s).
Add column name(s) for verification.
Provide additional parameters if required (e.g., Regex pattern, Python UDF).
Single vs. Combination Column Verification
The following verifications support both individual columns and column combinations:
Column Datatype Verification.
Column Duplicates Verification.
Column Not Null Verification.
Configuration Examples
Single column: Enter the column name and press Enter.
Multiple columns: Enter column names separated by commas and press Enter.
Step 4: Save or Run the Job
Choose an execution option:
Save – Saves job configuration for later execution.
Save and Start – Saves and immediately starts the evaluation.
Quick Run – Optimized execution for faster processing (supports up to 500,000 records).
Last updated