STARWEST 2026 - Exploratory Testing
Wednesday, September 23
Making Exploratory Testing Data-Driven with Pareto Analysis
This session presents a disciplined approach to exploratory testing that combines component-level defect analysis with focused and data-driven test charter design. Christopher will demonstrate how to decompose an application into meaningful components, consistently map defects to those components, and apply Pareto analysis to identify the areas responsible for the majority of defects. These high-risk components then become the basis for targeted exploratory test charters that summarize relevant defect history and provide testers with concrete test ideas and heuristics. Each exploratory...
Thursday, September 24
Testing in Production: How QA Frameworks Debug Life's Messy Systems
As QA professionals, we're experts at identifying system failures, analyzing root causes, and implementing sustainable fixes—at work. But when Alison McGuigan's personal life became a critical severity issue (layoff, miscarriage, postpartum fog, and a chaotic household), she realized she'd never applied that same rigor to her own circumstances. She treated her messy house like a failed deployment. So she ran an 8-week experiment: What if she debugged her life like a QA project? She conducted root cause analysis to find real problems beyond surface symptoms. She defined "minimum viable...
SLO-Driven Testing: Turning Reliability Targets into an Executable Test Strategy
Modern delivery pipelines still treat “testing” as something that happens before release, yet most high-impact failures in distributed systems are reliability failures that only show up under real traffic, real data, and real dependencies. In this session you will learn a practical, SLO-driven approach to unify quality engineering and reliability engineering. Shalini will start by translating critical customer journeys into a small set of measurable SLIs like latency, availability, error rate, and correctness signals and setting SLOs that reflect user expectations. Then she will walk...
RAG Testing That Holds Up: Evaluating LLMs for Faithfulness, Boundaries, and Trust
PreviewMany teams are adopting RAG to constrain LLMs to internal documents, policies, and knowledge bases, but “using RAG” does not guarantee trustworthy behavior. In practice, models still hallucinate, blend outside knowledge, ignore source boundaries, and produce confident answers that are not supported by retrieved evidence. Traditional test approaches (happy-path assertions, correctness spot checks, performance metrics) often miss these failures because the output reads plausibly correct. Drawing from real evaluation work on document-constrained enterprise systems, this session...