STARWEST 2024 - Quality Assurance

Generative AI (GenAI), exemplified by groundbreaking systems like ChatGPT and LLAMA, is revolutionizing the software landscape. These advanced technologies represent some of the most sophisticated software ever devised, capable of navigating an unprecedented range of prompts and questions, many of which have never been posed in human history. Their ability to generate varied responses to the same query and even fabricate answers when uncertain poses unique challenges in verification and testing. This talk delves into the intricacies of validating such systems and identifies areas needing...

In many teams, the total number of possible combinations of inputs, outputs, browsers, and devices for the software we need to test has grown to an alarming number. As testers, we need to choose the most important tests first, but how do we do that without understanding the potential scope in the first place? In this tutorial, Julie Gardiner will share two powerful testing techniques that can help us be more efficient and effective with our testing. Classification trees are a structured, visual approach to identifying test objects and documenting test ideas and data in a way that allows...

Companies of all sizes face difficulties in achieving higher quality standards within their organization. As quality engineering includes various roles and activities, it is also challenging to find the right people to analyze the current state from a wide perspective and provide the recommendations that will allow these organizations to mature their teams, improving their DevOps culture in the process. Join Péter Földházi as he illustrates the knowledge and skills necessary to become your company's quality consultant. Péter's goal is to involve engineers from beginner to advanced levels...

API testing has become more and more popular as service-oriented architectures have become common. In addition, testing at the API level can be effectively automated to provide maintainable regression tests that work well in a DevOps process. In this tutorial, Jeffery Payne discusses what API testing is all about and how AI is being leveraged today to make it easier to perform. Techniques and tools are discussed that highlight where in the testing process API testing makes the most sense. Various open-source and commercial tools will be demonstrated and the pros and cons of various...

Web applications are often security critical or serve as front-ends for security critical applications, making web testing for vulnerabilities an essential part of software testing. Unfortunately, most software testers have not been taught how to identify web security issues while testing applications. Join Tom Stiehm as he shares what you need to know to security test web-based applications as part of your overall testing process. Learn about the most common web security vulnerabilities and how they are introduced into web code and exploited by hackers. Explore test techniques for...

Everybody gets frustrated when web apps are broken, but testing them thoroughly doesn’t need to be a chore. Playwright, a new open-source browser automation tool from Microsoft, makes testing web apps fun! Playwright offers a slew of nifty features like automatic waiting, mobile emulation, and network interception. Plus, with isolated browser contexts, Playwright tests can set up much faster than traditional Web UI tests. In this session, we will automate concise yet robust web app tests for a Kanban board web app with Playwright in TypeScript. You'll learn to test modern web apps of any...

With the sudden rise of ChatGPT and large language models (LLMs), professionals have been attempting to use these types of tools to improve productivity. Building off prior momentum in AI for testing, software quality professionals are leveraging LLMs for creating tests, generating test scripts, automatically analyzing test results, and more. However, if LLM's are not fed good prompts describing the task that the AI is supposed to perform, their responses can be inaccurate and unreliable, thereby diminishing productivity gains. Join Tariq King as he teaches you how to craft high-quality AI...

Avoiding siloed development is a tricky business. It’s so easy for agile teams to fall into the rut of testers only doing traditional testing activities and programmers strictly doing their time-worn coding activities. But there are actually many places testers can be influential and contribute value on cross-functional teams. Robert Sabourin will share several examples of how testing skills can be applied to a wide variety of activities in an agile project. For instance, testers are among the most skilled team members in story grooming, elicitation, and exploration. Risk analysis in self-...

Jun-young Kwak (Sr. Director, Engineering), and Priyanka Halder (Sr. Leadership, QA & Release) will delve into the fascinating journey of Tinder’s quality engineering over the past decade, highlighting the dynamic evolution and innovative strategies that have underpinned the platform’s success in connecting millions worldwide. The talk traces the early days of experimentation with various quality assurance models, both in-house and outsourced, reflecting on the lessons learned and the pivotal decisions that shaped the future of quality at Tinder. A significant milestone in this journey...

What do you think about when you hear the words "test automation"? Are you considering developing your own tools to help you probe the product deeply, or will you apply a commercial tool for automated checking? Are you excited by new, powerful, enabling technologies, or do you worry that a robot will replace you? One thing is for sure: when we apply tools, we fundamentally change the context and the nature of the work we do.Our panelists are software testing experts who will discuss all this and more with you! Together, we'll probe what people mean by automation, the power that tools can...

We prepare for the future by learning from the lessons of the past. During this session, you will look back at some of the craziest bugs John Jenkins has run across during his career, and see what lessons can be gleaned from them to help tackle the problems of tomorrow. Bugs come in all shapes and sizes, and can exist in processes just as easily as they can exist in code. In this session, both types will be examined, including: the too much free space bug, the V1 bug, the too much test data bug, and more. The session will also explore some of the best practices John has developed in his...

In today's technology-driven landscape, where software and hardware intertwine seamlessly in embedded systems, adopting a DevOps culture becomes imperative. This talk delves into the dynamic world of embedded systems and explores how organizations can successfully implement and nurture a DevOps culture within this unique domain. DevOps, with its emphasis on collaboration, automation, and continuous improvement, has revolutionized software development. However, adapting these principles to the realm of embedded systems presents distinct challenges and opportunities. Our discussion will...

Delivering performance applications is a multi-layered challenge these days. Teams must assure that the application performs on multiple fronts. Each new function must have quick responses and be as slim as possible from the coding moment. It must be quick and light for a single request, handle concurrency, and coexist with multiple other processes even in micro-serviced applications spread all over the place. Not to mention, it must handle lots of loads once in production. It must stay performant and available within the agreed thresholds. The impressive amount of tasks to tackle and co-...

Testers are not part of the process; they don’t have sufficient knowledge of the client’s business to understand the big picture. This failure to understand leads to unsatisfactory outcomes that leave the client with unanswered questions. For a client to feel testing is of value, the software must work seamlessly and exceed their expectations. The solution is to shift testing left towards discovery and include testers early in the process so they have a stake in end-to-end results. Testers must understand the subject matter, ask questions, and be a valued part of the core team. Testing is...

The generation of acceptance criteria through generative AI is an innovative approach to streamline the requirements gathering process. However, ensuring that the implemented code aligns with these criteria is crucial for delivering high-quality software. This talk explores the integration of generative AI generating acceptance criteria with test case scenarios, aiming to establish a seamless connection between the development and testing phases. By leveraging pull requests as a central hub, this approach facilitates the validation of whether the test case scenarios adequately cover the...

Everyone involved in the SDLC knows quality is important. Absolutely. After all, who wants to push out (let alone use) software that doesn't work well? But, does your organization really embrace QA in all its glory? Do they see QA as more than ‘just’ testing? Do you see QA as more than ‘just’ testing? If your answer is 'No' to any of these questions, then this talk is for you. Yes, we test and log defects, but we also have to be diplomatic champions of quality assurance. We have to work to make sure our organizations see QA as a proactive function that spans the Software Development Life...

Join Vinod Kashid as he walks through his experience building end-to-end automation for performance testing and engineering. His team was tasked to do daily performance testing activities for a critical application and business executives were expecting detailed application performance outcomes on a daily basis. Manual performance test execution and analysis were time consuming processes and were delaying the overall run time for final report. Business stakeholders wanted to implement zero touch automation from performance test execution to analysis and reporting (including deep-dive...

As the lone tester amongst a sea of developers, Sylvia Solorzano struggled to scale reactive efforts into a cohesive program. Haphazard bug reports strained relationships and failed to provide direction. By installing a phased methodology based on industry best practices first, quality became her north star. Sylvia enshrined requirements traceability and manually executed test cases for each function delivered. Then she evaluated integrations between components and expanding to validate full system behavior as users would experience. Finally, leveraging test standards, she layered on...

Without training in software testing that includes formal methods such as equivalence class partitioning, boundary value analysis, decision tables, state diagrams, and others, an engineer, in good faith, will test the code intuitively until reaching “qualitative confidence” that testing is sufficient. The team at Trimble, Inc. provided training to engineers in a wide variety of formal testing methods so they could gain “quantitative confidence” that code has been tested sufficiently, by using methods known to achieve concrete coverage in requirements and code. Subsequently, Trimble...