Data Propagation

Overview

Data Propagation is the ability to capture data from one test step and automatically pass it to subsequent steps in your workflow. This enables realistic end-to-end testing where each step uses output from previous steps, creating complex scenarios that mirror actual user workflows.

---

Why Data Propagation Matters

Realistic Workflows

Tests flow like real users - output from one action becomes input to the next.

Data Dependencies

Reduced Test Data

Dynamic Testing

---

How Data Propagation Works

Basic Data Flow

Step 1: API Create User
  └── Response: {user_id: "123", email: "[email protected]"}

Step 2: API Login
  └── Input: email from Step 1
  └── Response: {token: "abc123def456"}

Step 3: Web Login
  └── Input: email and password
  └── Use token from Step 2 in header

Step 4: Web Verify Profile
  └── Use token from Step 2
  └── Verify user_id from Step 1 in profile

Data Variable Syntax

Reference previous step outputs using variable syntax:

${step_name.output_field} or ${step_number.output_field}

Examples:

• ${Step1.user_id}

• ${CreateUser.email}

• ${2.response.token}

---

Capturing Output Data

Types of Data to Capture

API Responses

• JSON fields: $.user.id, $.response.status

• Response headers: Authorization, Set-Cookie

• Status codes

• Response body

Web/Mobile Outputs

• Text values from elements

• Attribute values

• Page titles/URLs

• Form field values

Database Queries

• Query results

• Column values

• Record IDs

---

Extracting Data from Steps

Method 1: Automatic Extraction

Test scripts may automatically output data:

Example: API Step outputs
{
  "user_id": "12345",
  "email": "[email protected]",
  "token": "abc123"
}

Available for next steps as:
${CurrentStep.user_id}
${CurrentStep.email}
${CurrentStep.token}

Method 2: Manual Extraction

Specify which data to capture:

1. Open step configuration

2. Click "Extract Output" or "Capture Data"

3. Define extraction rules:

   • Field name

   • Source (response, header, etc.)

   • Path to value (JSON path, XPath, etc.)

4. Save configuration

Method 3: Data Hub

Store data in centralized location:

Data Hub Action
├── Store: Save output for later
├── Key: Unique identifier
└── Value: Data to store

Later retrieval:
├── Data Hub Action
├── Retrieve: Get stored data
└── Key: Use identifier

---

Using Captured Data

Example 1: User Creation and Login

Step 1: Create User (API)
  ├── POST /users
  ├── Response:
  │   ├── user_id: "u_12345"
  │   ├── email: "[email protected]"
  │   └── verification_code: "v_98765"
  └── Output stored

Step 2: Verify Email (API)
  ├── POST /users/${Step1.user_id}/verify
  ├── Body: verification_code: "${Step1.verification_code}"
  └── Confirms email with code from Step 1

Step 3: Login (Web)
  ├── Navigate to login page
  ├── Email: ${Step1.email}
  ├── Password: <predefined>
  └── Uses email from Step 1

Step 4: Verify Dashboard
  ├── Check username displayed
  ├── Expected: ${Step1.email}
  └── Verifies login was successful

Example 2: Payment Processing

Step 1: Create Order (API)
  ├── POST /orders
  ├── Response:
  │   ├── order_id: "ORD_789"
  │   └── total_amount: 99.99
  └── Output captured

Step 2: Process Payment (API)
  ├── POST /payments
  ├── Body:
  │   ├── order_id: "${Step1.order_id}"
  │   └── amount: "${Step1.total_amount}"
  └── Uses order data from Step 1

Step 3: Verify Order Status (Web)
  ├── Navigate to order: /orders/${Step1.order_id}
  ├── Verify amount: ${Step1.total_amount}
  └── Check order marked as paid

Step 4: Get Invoice (API)
  ├── GET /invoices?order_id=${Step1.order_id}
  ├── Verify invoice generated
  └── Extract invoice_id for next step

---

Data Mapping

Simple Mapping

Direct field-to-field mapping:

Step 1 Output: email
  ↓
Mapped To: Step2Input.username

Complex Mapping

Transform data during mapping:

Step 1 Output: { 
  "date": "2024-01-28T10:30:00Z"
}

Transformation:
  ${moment(Step1.date).format('MM/DD/YYYY')}
  
Result: 01/28/2024

Conditional Mapping

Use data conditionally:

IF ${Step1.user_type} == "admin"
  THEN use ${Step1.admin_token}
  ELSE use ${Step1.user_token}

---

Data Transformation

Available Transformations

Function	Purpose	Example
substring()	Extract part of string	${Step1.email.substring(0, 5)}
toUpperCase()	Convert to uppercase	${Step1.name.toUpperCase()}
toLowerCase()	Convert to lowercase	${Step1.name.toLowerCase()}
replace()	Replace text	${Step1.url.replace('http', 'https')}
split()	Split string	${Step1.data.split(',')[0]}
trim()	Remove whitespace	${Step1.text.trim()}
parseInt()	Convert to integer	${parseInt(Step1.count)}
Date functions	Date manipulation	${now()}, ${formatDate()}

Example Transformations

Extract ID from URL:
Input: "https://example.com/users/12345"
Transformation: ${Step1.url.split('/').pop()}
Output: "12345"

Create email from name:
Input: first_name = "john", last_name = "doe"
Transformation: ${Step1.first_name.toLowerCase()}.${Step1.last_name.toLowerCase()}@company.com
Output: "[email protected]"

Format phone number:
Input: "1234567890"
Transformation: (${Step1.phone.substring(0,3)}) ${Step1.phone.substring(3,6)}-${Step1.phone.substring(6)}
Output: "[phone]"

---

Accessing Nested Data

JSON Path Navigation

Access nested values using dot notation:

Response: {
  "user": {
    "profile": {
      "email": "[email protected]",
      "name": "John Doe"
    }
  }
}

Access:
${Step1.user.profile.email} → "[email protected]"
${Step1.user.profile.name} → "John Doe"

Array Access

Access array elements:

Response: {
  "results": [
    { "id": 1, "name": "Alice" },
    { "id": 2, "name": "Bob" }
  ]
}

Access:
${Step1.results[0].name} → "Alice"
${Step1.results[1].id} → 2
${Step1.results.length} → 2

---

Real-World Workflow Examples

Example 1: E-Commerce End-to-End

Complete Workflow: Purchase Flow

Step 1: Browse API - Get Products
  └── Outputs: product_id, price, stock

Step 2: Add to Cart API
  ├── Input: product_id from Step 1
  └── Outputs: cart_id, item_count

Step 3: Web - Login
  ├── Email: ${predefined}
  ├── Password: ${predefined}
  └── Outputs: session_token

Step 4: Web - Checkout
  ├── Headers: Authorization: ${Step3.session_token}
  ├── Item: ${Step1.product_id}
  ├── Quantity: 1
  └── Outputs: order_id

Step 5: Payment API
  ├── order_id: ${Step4.order_id}
  ├── amount: ${Step1.price}
  └── Outputs: transaction_id, confirmation

Step 6: Email Verification
  ├── Check inbox for order: ${Step4.order_id}
  ├── Verify amount: ${Step1.price}
  └── Confirm delivery expected date

Step 7: Web - Order History
  ├── Token: ${Step3.session_token}
  ├── Verify order: ${Step4.order_id}
  ├── Check status: Confirmed
  └── Verify amount: ${Step1.price}

Example 2: API Data Pipeline

Data Pipeline: ETL Workflow

Step 1: API - Extract Data
  ├── GET /source/data
  └── Outputs: [records array with 100+ items]

Step 2: Transform Data
  ├── Map: ${Step1.records}
  ├── Transform: uppercase names
  └── Outputs: transformed_data

Step 3: API - Validate Data
  ├── POST /validate
  ├── Body: ${Step2.transformed_data}
  └── Outputs: validation_results

Step 4: API - Load Data
  ├── POST /destination
  ├── Body: ${Step3.validated_records}
  └── Outputs: load_count, record_ids

Step 5: Verify Load
  ├── GET /destination/verify
  ├── Count: ${Step4.load_count}
  └── Confirm all records loaded

---

Best Practices

✅ Do

• Capture all relevant output data

• Use descriptive variable names

• Document data flow in comments

• Test data propagation with sample data

• Handle missing/null data gracefully

• Validate captured data format

• Use data transformation when needed

• Monitor data for sensitive information

❌ Don't

• Propagate sensitive data (passwords, keys) unnecessarily

• Assume data format (verify structure first)

• Create overly complex data transformations

• Forget to handle null/missing data

• Store unnecessary data

• Hardcode values when propagation available

• Propagate across unrelated workflows

• Leave sensitive data in logs

---

Troubleshooting

Issue: Variable not recognized

Symptoms:

• Variable shows as literal text: ${Step1.email}

• Step fails with "variable not found"

Solutions:

1. Verify step name or number is correct

2. Check field name spelling

3. Confirm output was actually captured

4. Review step execution to verify output

Issue: Data format incorrect

Symptoms:

• Data arrives in wrong format

• Type mismatch (string vs number)

• Transformation not working

Solutions:

1. Review data structure from source step

2. Add type conversion: ${parseInt(Step1.value)}

3. Test transformation logic

4. Use correct accessor path

Issue: Null or empty values

Symptoms:

• Variable is null or empty

• Step fails with null reference error

Solutions:

1. Add null checking: ${Step1.value || 'default'}

2. Verify previous step executed successfully

3. Check conditional logic for step execution

4. Review if step is being skipped

---

Integration with Other Features

• Session Persistence: Data persists across steps

• Action Types: Retry and data extraction work together

• Notifications: Include propagated data in alerts

• Reports: Show data flow and transformations

• Data Hub: Store and retrieve propagated data

---

Related Topics

• Workflows

• Nodes and Steps

• Execution Flow

• Data Hub Action

• Advanced: Custom Variables

• Advanced: Error Handling