STARWEST 2026 - Big Data, Analytics, AI/Machine Learning for Testing

Impostering a bit in the AI-verse? Overwhelmed by daily AI announcements? Unsure you're using AI most effectively? Tiny bit of FOMO? Jeremiah has you covered! In this workshop, he'll help you become an AI Power User. Become a boss at your job, whatever your role or industry! He'll show you where AI shines and where you'll want to be careful, plus toss you lots of hands-on practice. In the time together, Jeremiah will help you pinpoint YOUR niche, build a custom AI assistant, and develop a comms strategy to show off your new skills. You'll walk out with cutting-edge knowledge, a...

Are you tired of working on user stories that seem to be missing vital details for testing? Are you frustrated with being left out of vital design conversations? Or are you fed up with sizing estimates that never turn out to be true to reality? Then it’s time to stop guessing your way through product development and start planning it with better behavior discovery. In this tutorial, we will learn how three vital roles – business, development, testing – can collaborate on what features to build and test through the structured activities of story mapping and example mapping. We will practice...

The conversation around AI has already moved beyond prompts and chatbots. Today's engineering teams are beginning to build agents that can reason, use tools, and perform meaningful work. While many professionals have experimented with large language models (LLMs), far fewer understand how agents actually work or how to build one themselves. In this hands-on tutorial, you'll move beyond simply using AI and begin building with it. Through practical exercises and real-world examples, you'll learn the foundations of modern agentic systems and follow the evolution from LLM to agent. Along the...

Although there are several controversies and misunderstandings surrounding AI and machine learning, one thing is apparent — people have quality concerns about the safety, reliability, and trustworthiness of these types of systems. Not only are ML-based systems shrouded in mystery due to their largely black-box nature, they also tend to be unpredictable since they can adapt and learn new things at runtime. Validating ML systems is challenging and requires a cross-section of knowledge, skills, and experience from areas such as mathematics, data science, software engineering, cyber-security,...

You have the basics of AI tools covered, now let’s push them to the limits. In this advanced, hands-on workshop, we’ll go beyond vibe-coding and explore how to use AI tooling as a strategic orchestration tool for test automation engineers. Suitable for QA Engineers, Test Automation Engineers, Frontend Developers, and DevOps Engineers; we’ll dive deep into advanced techniques like multi-agent reasoning for debugging, building robust end-to-end tests, maintaining long-context conversations without drift, and crafting reusable automation patterns. We’ll be building custom tooling and agents...

AI agents have existed for decades, but generative AI has fundamentally changed what agents can do and how they are designed and built. Come and explore the evolution of AI agents across two major waves. First, learn the foundations of Agentic AI through agents built using rules, heuristics, and traditional machine learning, examining where these approaches excel and why they struggle with complexity, ambiguity, and scale. Then dive into the second wave of agents powered by generative AI and multimodal large language models. These modern agents can reason, plan, use tools, and interact...

In this tutorial, you’ll learn how to use GenAI to implement and test a REST API from scratch. Using Cursor as your development environment, you’ll be guided through a hands-on experience that combines powerful tools like Mocha, Supertest, k6, and GitHub Actions to implement automated testing and continuous integration. Under the guidance of Julio de Lima, you’ll first dive into essential REST API architecture concepts to build a solid foundation. Then, with the support of GenAI, you’ll generate and refine your API, create functional test cases, and automate them to validate behavior and...

Ready to push Playwright to the next level? This hands-on coding tutorial targets developers, testers, and SDETs who already know Playwright basics and want to scale their automation using AI-driven workflows. Using Playwright with TypeScript, we’ll dive straight into coding with tools like Playwright CLI and MCP, alongside AI coding agents such as Claude, Cursor, Copilot, or Codex. You’ll learn how to plan tests with AI, generate new coverage, and heal broken tests efficiently, while structuring your code with page objects and managing test data to avoid collisions. We’ll also explore...

AI is in the systems you're testing — and the rules just changed. With the EU AI Act in full enforcement, 250+ U.S. state bills in motion, and audit demands rising, "we tested it" is no longer enough. Regulators, customers, and your own legal team want evidence: who validated what, against which risks, with what controls running when the model shipped. The good news: testers are the natural front line for AI governance. You already own validation, traceability, and the test-evidence chain that auditors will ask for. The opportunity is to make that work load-bearing for AI systems — and to...

With the sudden rise of ChatGPT and large language models (LLMs), professionals have been attempting to use these types of tools to improve productivity. Building off prior momentum in AI for testing, software quality professionals are leveraging LLMs for creating tests, generating test scripts, automatically analyzing test results, and more. However, if LLM's are not fed good prompts describing the task that the AI is supposed to perform, their responses can be inaccurate and unreliable, thereby diminishing productivity gains. Join Tariq King as he teaches you how to craft high-quality AI...

As Generative AI moves from prototypes to production, one truth is emerging across every major tech organization — scaling AI responsibly isn’t just a data, science or engineering challenge but also an orchestration challenge. Behind every AI system that ships reliably, there’s a hidden architecture of alignment: between model builders, applied scientists, data engineers, security, compliance, and product teams. In that architecture, Technical Program Management (TPM) is becoming the connective tissue — the execution layer of modern AI. In this talk, Raj Karan shares lessons from leading...

As AI-powered features (especially generative AI) are rapidly integrated into modern software, testing teams face a critical challenge. Traditional testing approaches focus on correctness and performance but fail to uncover ethical risks such as bias, hallucinations, and accessibility regressions. In real projects, this has led to AI systems that technically “work” yet exclude users, generate misleading outputs, or erode trust. In this talk, Aditi addresses this gap by reframing AI quality as a testable concern and applying practical, tester-led techniques rather than data science-heavy...

Accessibility testing is critical, yet often under-tested due to limited expertise, time constraints, and manual effort. While tools exist, teams still struggle to translate WCAG guidelines into actionable, repeatable automated checks. In this session, Sidhartha will explore how AI can responsibly assist accessibility testing — not by replacing standards or human judgment, but by bridging the gap between guidelines and executable tests. Using Playwright MCP together with an LLM CLI, Sidhartha will demonstrate how AI can: interpret WCAG requirements, generate meaningful accessibility test...

Generative AI has created a new risk for quality leaders: "Coverage Theater." This occurs when AI-generated test suites inflate code coverage metrics to record highs while silently reducing assertion quality, leaving teams with green dashboards but escaping defects. In this session, Niranjan will dismantle this illusion by implementing a Quality Governance Audit using two advanced metrics that reveal what coverage hides. He will introduce the Assertion Strength Index (ASI), a scoring framework that rates tests from generic "existence checks" to rigorous business validation, exposing GenAI’...

The Quality Nervous System is a biologically inspired network where AI agents and humans operate symbiotically in a single adaptive system. AI agents continuously explore, learn, and execute in real time across software at machine speed, while humans provide the judgment, strategy, and purpose to assure outcomes align with user and business goals. AI partnering fundamentally changes how software is built. Humans now collaborate with systems that generate code, tests, insights, and behavior at unprecedented speed and volume. Continuous real-time results flood teams faster than they can...

As organizations increasingly deploy AI and machine learning systems into production, testing practices built for static, rule-based software are no longer sufficient. Unlike traditional applications, AI systems learn from data, change behavior over time, and are sensitive to data drift, bias, and feedback loops, making defects harder to detect with conventional test cases. This session presents a practical, experience-driven approach to testing AI systems across the full lifecycle, from model development to live deployment. Drawing on real-world implementations and applied research, the...

As delivery cycles compress, single AI agents are not enough. The next leap is a coordinated swarm of specialized QA agents, each owning a slice of the quality lifecycle (requirements, test generation, execution intelligence, defect triage, and release decisions). This session shows how to design an agent operating model that scales across teams, products, and pipelines without losing trust, traceability, or control. This session will introduce a practical blueprint for deploying multiple cooperating AI agents across the SDLC, with clear boundaries, KPIs, and governance that align to...

Many teams are adopting RAG to constrain LLMs to internal documents, policies, and knowledge bases, but “using RAG” does not guarantee trustworthy behavior. In practice, models still hallucinate, blend outside knowledge, ignore source boundaries, and produce confident answers that are not supported by retrieved evidence. Traditional test approaches (happy-path assertions, correctness spot checks, performance metrics) often miss these failures because the output reads plausibly correct. Drawing from real evaluation work on document-constrained enterprise systems, this session...