Testing AI Systems That Learn in Production: From Static Test Cases to Continuous Validation
As organizations increasingly deploy AI and machine learning systems into production, testing practices built for static, rule-based software are no longer sufficient. Unlike traditional applications, AI systems learn from data, change behavior over time, and are sensitive to data drift, bias, and feedback loops, making defects harder to detect with conventional test cases. This session presents a practical, experience-driven approach to testing AI systems across the full lifecycle, from model development to live deployment. Drawing on real-world implementations and applied research, the talk explores how to validate training data, monitor model behavior post-release, detect performance degradation, and test AI systems that adapt continuously in production. Attendees will learn how to design effective validation pipelines, define meaningful quality metrics for AI, and integrate monitoring and testing into CI/CD workflows without slowing delivery. The session emphasizes practical techniques teams can apply immediately, including data-centric testing, feedback-loop validation, and risk-based test strategies tailored for AI-driven products. Participants will leave with a clear framework for evolving their test practices to ensure reliability, trust, and quality in AI-powered systems.
Surya Narayana Reddy Chintacunta is a data and AI engineer and independent researcher specializing in building and validating scalable machine learning systems in production environments. His work focuses on bridging the gap between AI research and real-world deployment, with particular emphasis on data-centric design, continuous validation, and system reliability. Surya has published peer-reviewed research on AI-driven optimization, real-time feature engineering, and adaptive machine learning systems, and he actively serves as a peer reviewer for journals and conferences in software engineering and applied AI. With experience collaborating across engineering, data, and product teams, he brings a practical perspective on testing and maintaining AI systems that evolve over time under real-world constraints.
Sowjanya Deva is a Data Engineer and AI researcher with expertise in data engineering, machine learning, MLOps, distributed systems, and large language models (LLMs). Her research focuses on scalable AI infrastructure, parameter-efficient fine-tuning, privacy-preserving machine learning, and real-time feature engineering for next-generation AI systems. She is the author of the published paper Privacy-Preserving Customer Segmentation for Scalable Media Optimization in E-Commerce and the paper Scalable Real-Time Feature Engineering Pipelines for Large Language Model Training: A Distributed Systems Approach. Her accepted work, Rethinking MLOps: Building Modular, Serverless Machine Learning Pipelines on AWS, is forthcoming. Sowjanya actively contributes to the AI community as a peer reviewer, editorial board member, hackathon judge, and volunteer mentor, supporting research excellence and innovation in artificial intelligence, data engineering, and analytics.