STARWEST 2026 - Test Strategy, Planning, Metrics
Sunday, September 20
Software Tester Certification Foundation Level—ISTQB CTFL v4.0
Fundamentals of Agile Test Automation—ICAgile Certification (ICP-ATA)
Fundamentals of AI—ICAgile Certification (ICP-FAI)
Monday, September 21
Stop Guessing and Start Planning with Better Behavior Discovery
Are you tired of working on user stories that seem to be missing vital details for testing? Are you frustrated with being left out of vital design conversations? Or are you fed up with sizing estimates that never turn out to be true to reality? Then it’s time to stop guessing your way through product development and start planning it with better behavior discovery. In this tutorial, we will learn how three vital roles – business, development, testing – can collaborate on what features to build and test through the structured activities of story mapping and example mapping. We will practice...
Becoming an AI-Native Testing Organization
NewAI is changing how software is designed, built, and validated. As industries transition to AI-native product development, testing organizations must adapt their practices and skills. Manual testing is no longer enough; traditional automation should be enhanced with AI-driven quality engineering, autonomous agents, and data-powered tactics for faster and more reliable product delivery. Join Adam Auerbach to explore what it means to become an AI-Native Testing Organization. He will outline the AI-native software development lifecycle (SDLC) and highlight necessary changes in quality...
Smarter Test Design with Classification Trees and Pairwise Techniques
In many teams, the total number of possible combinations of inputs, outputs, browsers, and devices for the software we need to test has grown to an alarming number. As testers, we need to choose the most important tests first, but how do we do that without understanding the potential scope in the first place? In this tutorial, Julie Gardiner will share two powerful testing techniques that can help us be more efficient and effective with our testing. Classification trees are a structured, visual approach to identifying test objects and documenting test ideas and data in a way that allows...
Automation Framework Essentials
Automation is critical in today’s software delivery lifecycle, and yet many organizations struggle to keep their automation running. How can we mitigate difficulties and get consistent automation runs and results we can trust? The secret is implementing a solid automation framework, but that isn’t as easy as it seems. Chris Loder has built several automation frameworks over his career and has learned what works—and, more importantly, what doesn’t. This tutorial will cover what an automation framework is, the benefits of having one, and the keys to a successful framework, including...
Tuesday, September 22
Exploratory Testing in the Heat of the Sprint
Agile teams are burdened with the challenge of delivering working product increments after short iterations of development. Getting software from an ambiguous terse, incomplete requirement–to a done, working, solid, valuable, high-quality code requires testers to continuously adapt to change in a turbulent context and deliver actionable results. Chris Blain will illustrate how charter-driven session-based exploratory testing techniques can empower agile teams and help them learn quickly and adapt based on what really matters. Testers can design and execute tests on the fly as they explore...
Holistic Performance Testing for Modern Applications
With the advent of frameworks like Angular, React, and Vue, the landscape of application performance has changed significantly in terms of testing and measurement. Gone are the days of measuring response time as a single value based on back-end performance. In modern web and mobile applications, additional layers need to be peeled apart at the front end to truly understand its performance characteristics. Traditional approaches to performance testing are no longer sufficient to provide a delightfully responsive user experience. Join Kaushal Dalvi as he details new developments in the...
Accelerate Quality: A Hands-On Tutorial on AI-Assisted Software Testing
NewUnlock the next era of Quality Assurance (QA) by moving beyond simple code assistance and embracing the power of AI agents. This half-day, intensive tutorial offers hands-on experience with cutting-edge Generative AI tools, including GitHub Copilot and leading GenAI Chatbots, to integrate AI at every stage of the testing lifecycle. You will master practical techniques to dramatically accelerate quality, learning how to leverage AI to analyze requirements and identify risks, create comprehensive test cases and data, and accelerate test automation by generating scripts and suggesting...
From PRD to Production: Designing a Test Strategy That Actually Works
NewMost test strategies don’t fail in execution. They fail before testing even begins. They start too late, focus too narrowly on automation, and miss the one thing that actually matters: understanding what we are building and why. Janna and Cara will walk you through building a modern test strategy from the ground up, starting with the product requirements document (PDR) and carrying that intent through test design, execution, and measurement. They will break down a practical, end-to-end approach to quality strategy that connects product intent to engineering reality. You will learn how to...
Automating Test Design with a Little Help from Generative AI
NewRob Sabourin has spent over four decades pioneering automated test design across a wide range of technology stacks. More recently, he’s been exploring the power, promise—and occasional perversity—of applying Generative AI to the challenges of test design. In this lively and hands-on tutorial, Rob shares practical lessons from his experience using Generative AI to address real-world testing problems. From success stories and failures to unexpected surprises, he offers a candid look at what works, what doesn’t, and why. You will explore a variety of proven test design techniques, including...
Quality and Testing Measures and Metrics
To be most effective, leaders—including development and testing managers, ScrumMasters, product owners, and IT managers—need metrics to help direct their efforts and make informed recommendations about the software’s release readiness and associated risks. Because one important evaluation activity is to “measure” the quality of the software, the progress and results of both development and testing must be measured. Collecting, analyzing, and using metrics are complicated because developers and testers often are concerned that the metrics will be used against them. Join Jeff Pierce as he...
Prompt Engineering for Software Quality Professionals
With the sudden rise of ChatGPT and large language models (LLMs), professionals have been attempting to use these types of tools to improve productivity. Building off prior momentum in AI for testing, software quality professionals are leveraging LLMs for creating tests, generating test scripts, automatically analyzing test results, and more. However, if LLM's are not fed good prompts describing the task that the AI is supposed to perform, their responses can be inaccurate and unreliable, thereby diminishing productivity gains. Join Tariq King as he teaches you how to craft high-quality AI...
Wednesday, September 23
Automating the Migration: Scaling Cypress to Playwright Migrations with AI-Driven Velocity
PreviewThe decision to migrate from Cypress to Playwright is often stalled by the sobering reality of the manual effort required to rewrite extensive test suites. Traditionally, this involves months of tedious refactoring and logic translation that drains engineering resources and delays critical innovation. In this session, Ryan Song reveals a high-velocity framework designed to automate the heavy lifting of framework transition using Generative AI. He will move beyond simple prompts to explore a structured AI pipeline capable of handling complex asynchronous logic, custom commands, and...
From Local to Cloud: Scaling Your Load Tests with AWS (Without Blowing the Budget)
Many teams begin load testing on a local machine or inside their own network, but quickly hit limits with CPU, bandwidth, realism, and scale. This session addresses the challenge of moving from local load testing to cloud-based execution in a practical, cost-conscious way using AWS. The session will walk through how to spin up EC2 instances as load generators, create and manage SSH keys, transfer and run tests remotely, and collect results without needing deep cloud expertise. You’ll learn how to use Spot Fleets to reduce costs, structure your test setup for repeatability, and safely...
Making Exploratory Testing Data-Driven with Pareto Analysis
This session presents a disciplined approach to exploratory testing that combines component-level defect analysis with focused and data-driven test charter design. Christopher will demonstrate how to decompose an application into meaningful components, consistently map defects to those components, and apply Pareto analysis to identify the areas responsible for the majority of defects. These high-risk components then become the basis for targeted exploratory test charters that summarize relevant defect history and provide testers with concrete test ideas and heuristics. Each exploratory...
Beyond Coverage: Governing GenAI-Generated Tests with Metrics Leaders Can Trust
Generative AI has created a new risk for quality leaders: "Coverage Theater." This occurs when AI-generated test suites inflate code coverage metrics to record highs while silently reducing assertion quality, leaving teams with green dashboards but escaping defects. In this session, Niranjan will dismantle this illusion by implementing a Quality Governance Audit using two advanced metrics that reveal what coverage hides. He will introduce the Assertion Strength Index (ASI), a scoring framework that rates tests from generic "existence checks" to rigorous business validation, exposing GenAI’...
Testing AI Systems That Change Over Time
Modern software systems increasingly rely on AI-driven features such as recommendations, copilots, and automated decision-making. Unlike traditional software, these systems evolve over time as data changes and user behavior shifts, making them difficult to test using deterministic test cases alone. Many testing teams struggle with unpredictable outputs, flaky tests, and failures that only appear after deployment. In this session, Dr. Longe will address the challenge of testing AI-enabled systems that change over time and explain how testers can adapt familiar testing principles to these...
The Quality Nervous System
The Quality Nervous System is a biologically inspired network where AI agents and humans operate symbiotically in a single adaptive system. AI agents continuously explore, learn, and execute in real time across software at machine speed, while humans provide the judgment, strategy, and purpose to assure outcomes align with user and business goals. AI partnering fundamentally changes how software is built. Humans now collaborate with systems that generate code, tests, insights, and behavior at unprecedented speed and volume. Continuous real-time results flood teams faster than they can...
Thursday, September 24
AI Enablement at Scale: How to Lead an Organization Through a Successful AI Transformation
As AI rapidly moves from experimentation to a core organizational capability, many companies struggle not with models or tools—but with transformation itself. In this talk, Svetlana Stogni shares real-world experiences of leading AI enablement transformations at the organizational level. You will learn how to assess AI readiness, review existing PDLC processes, run rapid assessments, and establish continuous health monitoring to guide sustainable change. The session covers practical approaches to AI tools and platform setup, driving adoption across teams, defining performance and...
Quality Made Modern: The 2026 CoE Glow‑Up
Traditional Testing Centers of Excellence, once built for control, standardization, and governance, are struggling to keep pace with today’s AI‑driven, platform‑centric engineering landscape. Many organizations face the same challenge: fragmented testing practices, tool sprawl, inconsistent automation maturity, and a CoE model that feels more like a bottleneck than a value engine. In this session, Sunita will walk through how a modernized CoE can flip that script by shifting from enforcement to enablement, embedding quality into platform engineering, leveraging observability for real‑time...
Testing Event-Driven Systems Without Losing Your Sanity: Practical Patterns for AWS Serverless and Asynchronous Workflows
Event-driven architectures promise speed and scale, but they also introduce testing pain: eventual consistency, non-deterministic timing, duplicated events, and failures that only appear in production. In this talk, Parthiban will share a practical, field-tested approach he has used while leading distributed teams building regulated FinTech workloads on AWS serverless components such as Lambda, EventBridge, Step Functions, SQS, and API Gateway. He’ll start with the common failure patterns that make traditional end-to-end testing brittle, slow, and expensive. Next, he will walk through how...
Testing AI Systems That Learn in Production: From Static Test Cases to Continuous Validation
As organizations increasingly deploy AI and machine learning systems into production, testing practices built for static, rule-based software are no longer sufficient. Unlike traditional applications, AI systems learn from data, change behavior over time, and are sensitive to data drift, bias, and feedback loops, making defects harder to detect with conventional test cases. This session presents a practical, experience-driven approach to testing AI systems across the full lifecycle, from model development to live deployment. Drawing on real-world implementations and applied research, the...
Testing the Untestable: How to Validate Cloud‑Dependent Features You Don’t Fully Own and Control
Today’s software relies on a collection of cloud services, shared platforms, and third‑party tools, many of which your teams don’t own, control, or even fully understand. Yet when something goes wrong, customers don’t blame the cloud provider or the external API. They blame your product. That puts testers in a tough spot: how do you ensure quality when key parts of the system are unpredictable, unavailable, or outside your team’s reach? This session explores how to build confidence in features that depend on other teams and the ever‑changing cloud. The session will look at practical ways...
Taming the Stochastic Beast: Building AI Evaluation Pipelines for GenAI Releases
If you've ever shipped a GenAI feature wondering “is this actually good enough?”, you're not alone. Traditional pass/fail QA breaks down when outputs are non-deterministic, and teams end up making release decisions based on subjective “vibe checks” rather than data. This session shows how Product Managers can partner with QA to replace intuition with a systematic AI evaluation pipeline. You'll learn how to define quality as measurable dimensions (groundedness, tone, helpfulness, safety), build a representative test set, and design rubrics that align product goals with engineering...
Testing in Production: How QA Frameworks Debug Life's Messy Systems
As QA professionals, we're experts at identifying system failures, analyzing root causes, and implementing sustainable fixes—at work. But when Alison McGuigan's personal life became a critical severity issue (layoff, miscarriage, postpartum fog, and a chaotic household), she realized she'd never applied that same rigor to her own circumstances. She treated her messy house like a failed deployment. So she ran an 8-week experiment: What if she debugged her life like a QA project? She conducted root cause analysis to find real problems beyond surface symptoms. She defined "minimum viable...
SLO-Driven Testing: Turning Reliability Targets into an Executable Test Strategy
Modern delivery pipelines still treat “testing” as something that happens before release, yet most high-impact failures in distributed systems are reliability failures that only show up under real traffic, real data, and real dependencies. In this session you will learn a practical, SLO-driven approach to unify quality engineering and reliability engineering. Shalini will start by translating critical customer journeys into a small set of measurable SLIs like latency, availability, error rate, and correctness signals and setting SLOs that reflect user expectations. Then she will walk...
RAG Testing That Holds Up: Evaluating LLMs for Faithfulness, Boundaries, and Trust
PreviewMany teams are adopting RAG to constrain LLMs to internal documents, policies, and knowledge bases, but “using RAG” does not guarantee trustworthy behavior. In practice, models still hallucinate, blend outside knowledge, ignore source boundaries, and produce confident answers that are not supported by retrieved evidence. Traditional test approaches (happy-path assertions, correctness spot checks, performance metrics) often miss these failures because the output reads plausibly correct. Drawing from real evaluation work on document-constrained enterprise systems, this session...