Beyond Coverage: Governing GenAI-Generated Tests with Metrics Leaders Can Trust
Generative AI has created a new risk for quality leaders: "Coverage Theater." This occurs when AI-generated test suites inflate code coverage metrics to record highs while silently reducing assertion quality, leaving teams with green dashboards but escaping defects. In this session, Niranjan will dismantle this illusion by implementing a Quality Governance Audit using two advanced metrics that reveal what coverage hides. He will introduce the Assertion Strength Index (ASI), a scoring framework that rates tests from generic "existence checks" to rigorous business validation, exposing GenAI’s tendency to produce syntactically safe but semantically weak assertions. Next, he will apply the Behavioral Uniqueness Ratio (BUR) to identify "test bloat," using mutation clustering to reveal when hundreds of AI-generated tests are actually verifying the exact same behavior. You will leave with a diagnostic toolkit, starter thresholds, and a 4-week rollout plan to prune redundancy, implement new CI gates, and redesign dashboards that track true risk reduction rather than vanity numbers.
Niranjan Maharajh is an engineering leader in digital healthcare and medtech. He is Director of Systems, Hardware, and Test Engineering at Johnson & Johnson Medtech and Owner & Principal of NOPMARK Consulting, delivering accredited training that helps engineering teams build reliable products without losing speed. He previously directed Systems Engineering and Verification & Validation at Carl Zeiss Meditec, leading R&D transformation and building cross-functional teams spanning user experience (human factors/usability), systems design engineering, and V&V. He later led engineering for hospital solutions at Proximie and taught in UCSC Silicon Valley Extension’s Medical Device Quality and Design certificate program. He holds an M.S. in Biomedical Engineering (UT Austin) and serves on the ASTQB Board of Directors.