🔗Data Source Connectors

The Data Testing platform supports a wide variety of data source connectors, enabling seamless integration with your entire data ecosystem.

Overview

Data sources are the foundation of any validation job. Each connector requires specific configuration details such as:

• 🔐 Authentication & Credentials

• 🖥️ Host Information & Connection Details

• 📝 Query Configuration

• 🔑 Optional Access Keys & Parameters

Before creating a job, ensure you have the necessary connection information for your target data source. Consult your database administrator or cloud provider for exact credentials and endpoint details.

---

🌐 APIs & Webhooks

Connect to REST APIs and webhook endpoints for real-time data validation.

Connector	Use Case
API Connector	Real-time API validation, REST endpoint testing

Configure API Connector →

---

🗄️ SQL Databases

Connect to traditional relational SQL databases for structured data validation.

Cloud Platforms

Managed cloud SQL services:

• Amazon Redshift - Data warehouse on AWS

• Snowflake - Cloud data platform

Browse Cloud SQL →

Enterprise Databases

Open Source

View All SQL Connectors →

---

🚀 NoSQL Databases

Connect to modern multi-model and document databases for flexible data validation.

Connector	Type	Best For
MongoDB	Document Database	Document-based validation
SurrealDB	Multi-Model Database	Complex, multi-type data validation

View All NoSQL Connectors →

---

💾 Blob Storage & File Systems

Connect to cloud storage and file systems for batch data validation.

Connector	Platform	Use Case
Amazon S3	AWS	S3 bucket data validation
Azure Data Lake Storage Gen2	Microsoft Azure	ADLS Gen2 file validation
SFTP	Any SFTP Server	Remote file system validation

View All Storage Connectors →

---

🔐 Security Considerations

Important Security Practices:

1. ✅ Store credentials securely using environment variables or secrets management

2. ✅ Use read-only database users when possible

3. ✅ Restrict network access to your data sources

4. ✅ Regularly audit and rotate credentials

5. ✅ Enable encryption in transit (TLS/SSL)

6. ✅ Never commit credentials to version control

---

📋 Connector Comparison

Feature	SQL	NoSQL	APIs	Blob Storage
Real-time Access	✅	✅	✅	⚠️ Limited
Authentication	Username/Password	Various	API Keys/OAuth	Cloud Credentials
Query Language	SQL, SurrealQL	Native Query	REST	File Paths
Large Data	✅	✅	⚠️ Paginated	✅
Streaming	⚠️ Limited	✅	✅	⚠️ Limited

---

Getting Started

1. Choose your connector type based on where your data resides

2. Gather connection details from your IT/DevOps team

3. Configure the connector in your Data Testing job

4. Validate the connection before running validations

5. Create your first validation job using the configured source

Need help? Check out our Demo Configurations to see real-world connector setups!

---

Quick Links

• API Connector Setup

• SQL Connectors Guide

• NoSQL Connectors Guide

• Blob Storage Guide

• Demo: DB to DB

• Demo: API Integration