- ETL Testing Tutorial
- ETL Testing - Home
- ETL Testing - Introduction
- ETL Testing - Tasks
- ETL vs Database Testing
- ETL Testing - Categories
- ETL Testing - Challenges
- ETL - Tester's Roles
- ETL Testing - Techniques
- ETL Testing - Process
- ETL Testing - Scenarios(Test Cases)
- ETL Testing - Performance
- ETL Testing - Scalability
- ETL Testing - Data Accuracy
- ETL Testing - Metadata
- ETL Testing - Data Transformations
- ETL Testing - Data Quality
- ETL Testing - Data Completeness
- ETL Testing - Backup Recovery
- ETL Testing - Automation
- ETL Testing - Best Practices
- ETL Testing - Interview Questions
- ETL Testing Useful Resources
- ETL Testing - Quick Guide
- ETL Testing - Useful Resources
- ETL Testing - Discussion
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
ETL Testing – Automation
ETL testing is mostly done using SQL scripts and gathering the data in spreadsheets. This approach to perform ETL testing is very slow and time-consuming, error-prone, and is performed on sample data.
Technical Challenge in Manual ETL Testing
Your ETL test team writes SQL queries to test data in a warehouse system and they need to execute them manually using a SQL editor and then put the data into an Excel spreadsheet and compare them manually. This process is time-consuming, resourceintensive, and inefficient.
There are various tools available in the market to automate this process. The most common ETL Testing tools are QuerySurge and Informatica Data Validation.
QuerySurge is a data testing solution designed for testing Big Data, Data Warehouses, and the ETL process. It can automate the entire process for you and fit nicely into your DevOps strategy.
The key features of QuerySurge are as follows −
It has Query Wizards to create test QueryPairs fast and easily without the user having to write any SQL.
It has a Design Library with reusable Query Snippets. You can create custom QueryPairs as well.
It can compare data from source files and data stores to the target Data Warehouse or Big Data store.
It can compare millions of rows and columns of data in minutes.
It allows the user to schedule tests to run (1) immediately, (2) any date/time, or (3) automatically after an event ends.
It can produce informative reports, view updates, and auto-email results to your team.
To automate the entire process, your ETL tool should start QuerySurge through command line API after the ETL software completes its load process.
QuerySurge will run automatically and unattended, executing all tests and then emailing everyone on the team with results.
Just like QuerySurge, Informatica Data Validation provides an ETL testing tool that helps you to accelerate and automate the ETL testing process in the development and production environment. It allows you to deliver complete, repeatable, and auditable test coverage in less time. It requires no programming skills!