Agents All The Way Down
Are you building generative AI agents but losing sleep over how to ensure they behave correctly? As AI agents proliferate across domains from customer service to code generation, one challenge towers above the rest: how do we systematically test these unpredictable, creative systems for correctness? Traditional testing approaches fall short when your software can generate novel responses you've never seen before. Join Dionny Santiago as he explores the cutting edge of AI agent testing. See real examples across different domains, discover emerging validation techniques, and learn why testing an agent is fundamentally different. But here's where it gets interesting – what if the best way to test an agent is with another agent? This recursive approach is already reshaping quality assurance in surprising ways. Come discover practical strategies for taming your agents and a provocative vision of where AI testing is headed. Spoiler alert: the testers of tomorrow might not be human at all.
Dionny Santiago is an Engineering Manager at Indeed and has 12+ years of industry experience working as an SW/QA/ML engineer, software/test architect, and engineering manager. Dionny completed his master's thesis on exploring the intersection between AI and test automation and is currently pursuing a Ph.D. in this problem space. One of Dionny's goals is to advance the current state of the art in software testing through Artificial Intelligence and Machine Learning. He has published and contributed to the design of test specification languages and AI-driven test generation approaches. Dionny also enjoys attending software testing conferences to both learn and share. Dionny is a member of the IEEE Computer Society and the ACM.
