Validation of Data Pipelines with Test Automation
Good data today does not equal good data tomorrow. In today’s dynamic digital environments, traditional data testing methods are not enough. Manual “stare and compare” data checks are painfully slow and can’t scale.
Tricentis Data Integrity brings the discipline of end-to-end, automated, and continuous testing to the world of data — enabling you to drive higher data quality and better business outcomes at scale.
Come learn how to add data integrity test automation in your ETL, ELT and data pipelines. The intended audience is devops/dataops engineers, quality engineers, developers, testers, solutions architects, and anyone who is stuck manually comparing reports.
As you scale and grow your use of data, keeping on top of the steps can be difficult. Apache Airflow is an open-source orchestration tool that helps you programmatically create workflows that will help you run, schedule, monitor, and manage data engineering pipelines - no more manually managing those cron jobs! In this breakout session, Andrew Prueser of Tricentis will showcase how one can incorporate modern quality engineering approaches into your data workflows to automate the development, testing, and deployment of data pipelines via Apache Airflow.
Andrew Prueser has 20+ years of experience in high-tech manufacturing where he developed industry leading analysis software, spearheaded global data, systems integration, modernization, and M&A initiatives. In those years he has seen firsthand how good, or non-existent testing strategies can make or break a project. As a Solutions Architect now with Tricentis he is utilizing his development, and systems integration background to help teams recognize the benefits, and rewards of an integrated testing philosophy in their CI/CD pipelines, as well as in their data pipelines. He's on a mission to hike in all the National Parks (16 and counting), and to have visited more countries than states (currently 26/19 in favor of states).