STARWEST 2024 - Test Techniques

Many organizations invest a lot of effort in test automation at the system level but then have serious problems as their product matures and changes over time. As a leader, how can you ensure that your new automation efforts will get off to a good start? What can you do to ensure that your automation work provides continuing added value? Chris Loder will explain the critical issues you need to know to get a good start, and he will share his extensive experience in building great automation. He covers the most important management issues you should address for test automation success,...

API testing has become more and more popular as service-oriented architectures have become common. In addition, testing at the API level can be effectively automated to provide maintainable regression tests that work well in a DevOps process. In this tutorial, Jeffery Payne discusses what API testing is all about and how AI is being leveraged today to make it easier to perform. Techniques and tools are discussed that highlight where in the testing process API testing makes the most sense. Various open-source and commercial tools will be demonstrated and the pros and cons of various...

We prepare for the future by learning from the lessons of the past. During this session, you will look back at some of the craziest bugs John Jenkins has run across during his career, and see what lessons can be gleaned from them to help tackle the problems of tomorrow. Bugs come in all shapes and sizes, and can exist in processes just as easily as they can exist in code. In this session, both types will be examined, including: the too much free space bug, the V1 bug, the too much test data bug, and more. The session will also explore some of the best practices John has developed in his...

In today's dynamic business landscape, organizations grapple with the challenge of innovating across multiple product developments simultaneously. This session explores the intricacies of concurrent development involving seven products, highlighting the pivotal roles of scrum teams, engineering approach, and collaborative testing strategies. Traditional independent product testing approaches prove insufficient in the context of interdependence and a unified platform. The discussion centers on the need for an evolved testing strategy that shifts left, meticulously addressing integration...

Using artificial intelligence to generate test code is a hybrid automation strategy that combines the best of both worlds. Tests can be created very quickly by almost anyone using AI, yet the tests are still planned by humans and maintainable by humans. With the right prompts, you can have AI construct traditional test code using open source testing tools that the world is already familiar with (Chai, Mocha, Cypress). As a result, you end up with structured code that is logical and easy to maintain without having to wonder what the AI is testing. In this session, Timothy will look at...

As the lone tester amongst a sea of developers, Sylvia Solorzano struggled to scale reactive efforts into a cohesive program. Haphazard bug reports strained relationships and failed to provide direction. By installing a phased methodology based on industry best practices first, quality became her north star. Sylvia enshrined requirements traceability and manually executed test cases for each function delivered. Then she evaluated integrations between components and expanding to validate full system behavior as users would experience. Finally, leveraging test standards, she layered on...

As you research the latest in generative AI technology, you will see that the development and availability of multi-modal GPT engines are fundamentally changing the way applications are tested and described. These new GPT models can generate and interpret voice, text, and images seamlessly. For example, you can ask them to navigate an application to accomplish a business task and comment on their actions. This means that we’re for the first time entering the world of AI-assisted/performed exploratory testing. When you couple this with the capabilities of GPT models to identify UI elements...

Without training in software testing that includes formal methods such as equivalence class partitioning, boundary value analysis, decision tables, state diagrams, and others, an engineer, in good faith, will test the code intuitively until reaching “qualitative confidence” that testing is sufficient. The team at Trimble, Inc. provided training to engineers in a wide variety of formal testing methods so they could gain “quantitative confidence” that code has been tested sufficiently, by using methods known to achieve concrete coverage in requirements and code. Subsequently, Trimble...