Testing with the Coding Agent

Software testing is moving inside AI coding agents. These agents can generate code, tests, environments, data, reviews, and fixes in the same continuous loop, but that creates a dangerous question: should the system that wrote the bug also decide whether the software is ready to ship? Jason Arbon explores how testing changes when software is generated dynamically, requirements evolve through prompts, and agents work autonomously for hours or days. You will learn how to separate generation from independent validation, capture evidence from agent trajectories, test changing prompts and models, control tool permissions and side effects, and preserve human judgment at critical release boundaries. The session introduces the new AI Testing Pyramid, which expands testing beyond unit and integration checks to include behavioral evals, agent workflows, model variability, production evidence, and confidence engineering. The coding agent is becoming the new development environment, test harness, and release pipeline. Come learn how to make it earn your confidence.