Large-scale and complex testing projects can stress the testing and automation practices we have learned through the years, resulting in less than optimal outcomes. However, a number of innovative ideas and concepts are emerging to better support industrial-strength testing for big projects. Hans Buwalda shares his experiences and presents strategies for organizing and managing testing on large projects. Learn how to design tests specifically for automation, including how to incorporate keyword testing and other techniques. Learn what roles virtualization and the cloud can play—and the potential pitfalls of such options. Take away tips and tricks to make automation more stable, and to deal with the numerous versions and configurations common in large projects. Hans also describes the main challenges with global teams including time zones and cultural differences, and offers seven common problem "patterns" in globalization and what you can do to address them.More Information
Begin your experience by attending half- or full-day tutorials. Please note that you must register for the tutorial(s) you want to attend as space is limited and many sell out quickly.
You're under tight time pressure and have barely enough information to proceed with testing. How do you test quickly and inexpensively, yet still produce informative, credible, and accountable results? Rapid Software Testing, adopted by context-driven testers worldwide, offers a field-proven answer to this all-too-common dilemma. In this one-day sampler of the approach, Michael Bolton introduces you to the skills and practice of Rapid Software Testing through stories, discussions, and "minds-on" exercises that simulate important aspects of real testing problems. The rapid approach isn't just testing with speed or a sense of urgency; it's mission-focused testing that eliminates unnecessary work, assures that the most important things get done, and constantly asks how testers can help speed up the successful completion of the project. Join Michael to learn how Rapid Testing focuses on both the mind set and skill set of the individual tester, using tight loops of exploration and critical thinking skills to help continuously re-optimize testing to match clients' needs and expectations.
Participants are strongly encouraged to bring a Windows-compatible computer to the class.More Information
Whether you are new to testing or looking for a better way to organize your test practices, understanding risk is essential to successful testing. Dale Perry describes a general risk-based framework—applicable to any development lifecycle model—to help you make critical testing decisions earlier and with more confidence. Learn how to focus your testing effort, what elements to test, and how to organize test designs and documentation. Review the fundamentals of risk identification, analysis, and the role testing plays in risk mitigation. Develop an inventory of test objectives to help prioritize your testing and translate them into a concrete strategy for creating tests. Focus your tests on the areas essential to your stakeholders. Execution and assessing test results provide a better understanding of both the effectiveness of your testing and the potential for failure in your software. Take back a proven approach to organize your testing efforts and new ways to add more value to your project and organization.More Information
Selenium is an open source automation tool for test driving browser-based applications. WebDriver, the newly-introduced API for Selenium against which tests are written in Java, contains classes including ChromeDriver, AndroidDriver, and iPhoneDriver. Sometimes test authors find the API daunting and their initial automation code brittle and poorly structured. In this introduction, Alan Richardson provides hints and tips gained from his years of experience both using WebDriver and helping others improve their use of the tool. Alan starts at the beginning, explaining the basic WebDriver API capabilities—simple interrogation and navigation—and then moves on to synchronization strategies and working with AJAX applications. He covers tools and location strategies to find elements on web pages using CSS and XPath. Alan provides an introduction to abstraction approaches which help you build robust, reliable, and maintainable automation suites.
Hands-on exercises require a laptop computer with Firefox, Firebug, and Firepath installed. You will write code! Coding exercises require an IDE (IntelliJ), Java SDK, and Maven. Prior to the session, follow the Getting Started Guide at http://seleniumsimplified.com/get-started. Come ready to learn.
Have you ever needed a way to measure your leadership IQ? Or been in a performance review where the majority of time was spent discussing your need to improve as a leader? If you have ever wondered what your core leadership competencies are and how to build on and improve them, Jennifer Bonine shares a toolkit to help you do just that. This toolkit includes a personal assessment of your leadership competencies, explores a set of eight dimensions of successful leaders, provides suggestions on how you can improve competencies that are not in your core set of strengths, and describes techniques for leveraging and building on your strengths. These tools can help you become a more effective and valued leader in your organization. Exercises help you gain an understanding of yourself and strive for balanced leadership through recognition of both your strengths and your “development opportunities.”More Information
In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. With specific examples from recent client implementations, Scott Barber shares the fundamentals of implementing T4APM™, a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The T4APM™ approach hinges on applying a simple and unobtrusive Target, Test, Trend, Tune cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.More Information
To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Rick Craig as he addresses common metrics—measures of product quality, defect removal efficiency, defect density, defect arrival rate, and testing status. Learn the guidelines for developing a test measurement program, rules of thumb for collecting data, and ways to avoid “metrics dysfunction.” Rick identifies several metrics paradigms and discusses the pros and cons of each.
Delegates are urged to bring their metrics problems and issues for use as discussion points.
The practice of agile software development requires a clear understanding of business needs. Misunderstanding requirements causes waste, slipped schedules, and mistrust within the organization. Jared Richardson shows how good acceptance tests can reduce misunderstanding of requirements. A testable requirement provides a single source that serves as the analysis document, acceptance criteria, regression test suite, and progress-tracker for any given feature. Jared explores the creation, evaluation, and use of testable requirements by the business and developers. Learn how to transform requirements into stories—small units of work—that have business value, small implementation effort, and easy to understand acceptance tests. This tutorial features an interactive exercise that starts with a high level feature, decomposes it into stories, applies acceptance tests to those stories, and estimates the stories for business value and implementation effort. The exercise demonstrates how big requirement stories can be decomposed into business-facing stories, rather than into technical tasks that the business does not understood.More Information
You’ve “gone agile” and have been relatively successful. So, how do you know how well your team is really doing? And how do you continuously improve your practices? When things get rocky, how do you handle the challenges without reverting to old habits? You realize that the path to high-performance agile testing isn’t easy or quick. It also helps to have a guide. So consider this workshop your guide to ongoing, improved, and sustained high-performance. Join Bob Galen and Mary Thorn as they share lessons from their most successful agile testing transitions. Explore actual team case studies for building team skills, embracing agile requirements, fostering customer interaction, building agile automation, driving business value, and testing at-scale—all building agile testing excellence. Examine the mistakes, adjustments, and the successes, and learn how to react to real-world contexts. Leave with a better view of your team’s strengths, weaknesses, and where you need to focus to improve.More Information
As test managers and test professionals we can have an enormous emotional impact on others. We're constantly dealing with fragile egos, highly charged situations, and pressured people playing a high-stakes game under conditions of massive uncertainty. We're often the bearers of bad news and are sometimes perceived as critics, activating people's primal fear of being judged. Emotional intelligence (EI), the concept popularized by Harvard psychologist and science writer Daniel Goleman, has much to offer test managers and testers. Key EI skills include self-awareness, self-management, social awareness, and relationship management. Explore the concept of EI, assess your own levels of EI, and look at ways in which EI can help. Thomas McCoy discusses how EI can be useful in dealing with anger management, controlling negative thoughts, processing constructive criticism, and dealing with conflict—all within the context of the testing profession. This lively session is grounded in real-life examples, giving you concrete ideas to take back to work.More Information
The reality is that technology is complicated. As testers, we are challenged with complicated problems that need solving. Andy Glover presents a hands-on workshop that describes a new way of looking at testing problems and ideas. Andy demonstrates how thinking with pictures can help testers discover and develop new ideas, solve problems in unexpected ways, and dramatically improve their ability to share their insights with others. Andy shows how to clarify a problem or sell an idea by visually breaking it down using a set of visualization tools including mind maps, work flows, and powerful but simple visuals to communicate complex messages. You don’t need to know how to draw to attend this workshop. Although we are naturally creative, sometimes it takes effort to develop those skills. Join Andy to learn how to apply visual solutions to our everyday software testing challenges.More Information
Innovation is a word frequently tossed around in organizations today. The standard clichés are do more with less and be creative. Companies want to be innovative but often struggle with how to define, implement, prioritize, and track their innovation efforts. Using the Innovation to Types model, Jennifer Bonine will help you transform your thinking regarding innovation and understand if your team and company goals match their innovation efforts. Learn how to classify your activities as "core" (to the business) or "context" (essential, but non-revenue generating). Once you understand how your innovation activities are related to revenue generating activities, you can better decide how much of your effort should be spent on core or context activities. Take away tools including an Innovation to Types model for classifying innovation, a Core and Context model to classify your activities, and a way to map your innovation initiatives to different contexts.More Information
Testers want to be responsible and professional. However, they often come under pressure to comply with rules, standards, and processes that aren't always helpful. It's the price of keeping your auditors happy. But do you really know what auditors want? Are they all simply rule-obsessed, pedantic “little dictators”? James Christie shows why good auditors worry about risk—not rules. They want to explain the important risks to the people who lose sleep over them. James explains auditors' and regulators' attitudes toward risk and evidence. He shows that auditors' standards and governance models do have useful advice—knowledge that can help you choose the right testing approach for your project. James shows how to enlist smart auditors as valuable allies—and how to challenge the poor ones. Understanding auditors' needs will help you do better testing, at less cost. Wouldn't senior management and your stakeholders be interested in that?More Information
Automating system level test execution can result in many problems. It is surprising to find that many people encounter the same problems yet are unaware of common solutions that worked well for others. These problem/solution pairs are called “patterns.” Seretta Gamba recognized the commonality of these test automation issues and their solutions and, together with Mark Fewster, has organized them into Test Automation Patterns. Although unit test patterns are well known, Seretta’s and Mark’s patterns address more general issues. They cover management, process, design, and execution patterns to help you recognize common test automation issues and show you how to identify appropriate patterns to solve the problems. Issues such as No Previous Automation, High ROI Expectations, and High Test Maintenance Cost are addressed by patterns such as Maintainable Testware, Tool Independence, and Management Support.
Bring your laptop to gain access to an offline version of the wiki during the tutorial.More Information
Exploratory testing is an approach to testing that emphasizes the freedom and responsibility of testers to continually optimize the value of their work. Exploratory testing is the process of three mutually supportive activities—learning, test design, and test execution—done in parallel. With skill and practice, exploratory testers typically uncover an order of magnitude more problems than when the same amount of effort is spent on procedurally-scripted testing. All testers conduct exploratory testing in one way or another, but few know how to do it systematically to obtain the greatest benefits. Even fewer can articulate the process. Paul Holland shares specific heuristics and techniques of exploratory testing that will help you get the most from this highly productive approach. Paul focuses on the skills and dynamics of exploratory testing, and how it can be combined with scripted approaches.More Information
Jon Bach has been in testing for twenty years. Is testing in 2014 different from testing in 1994? If you’ve been in the business that long, maybe you’ve seen it move from a little bit of automation and tooling to almost total automation and tooling. Maybe you’ve seen lab setups go from hours of loading OS images on “boat anchor” boxes to virtual, on-demand, scalable cloud provisioning in seconds. Maybe you think testing is dead because we live in a DevOps world where it’s good enough to run a happy path checklist. Maybe you think testing isn’t dead because you’ve seen recent computer science graduates know dangerously nothing about the craft of sapient testing. Jon wants to know what you’ve seen in your careers. In exchange for his thoughts, he wants to hear yours: How do you define testing? What are your best ideas? What works for you? Under what contexts does testing not matter at all? Whether you’re stuck or confused, inspired or hopeful, come, listen, and contribute your experiences … On Testing.More Information
Critical thinking is the kind of thinking that specifically looks for problems and mistakes. Regular people don't do a lot of it. However, if you want to be a great tester, you need to be a great critical thinker. Critically thinking testers save projects from dangerous assumptions and ultimately from disasters. The good news is that critical thinking is not just innate intelligence or a talent—it's a learnable and improvable skill you can master. Michael Bolton shares the specific techniques and heuristics of critical thinking and presents realistic testing puzzles that help you practice and increase your thinking skills. Critical thinking begins with just three questions—Huh? Really? and So?—that kick start your brain to analyze specifications, risks, causes, effects, project plans, and anything else that puzzles you. Join Michael for this interactive, hands-on session and practice your critical thinking skills. Study and analyze product behaviors and experience new ways to identify, isolate, and characterize bugs.
Participants are strongly encouraged to bring a Windows-compatible computer to the class.More Information
Many organizations invest substantial time and effort in test automation but do not achieve the significant returns they expected. Some blame the tool they used; others conclude test automation just doesn't work in their situation. The truth, however, is often very different. These organizations are typically doing many of the right things but they are not addressing key issues that are vital to long term test automation success. Describing the most important issues that you must address, Mark Fewster helps you understand and choose the best approaches for your organization—no matter which automation tools you use. We’ll discuss both management issues—responsibilities, automation objectives, and return on investment—and technical issues—testware architecture, pre- and post-processing, and automated comparison techniques. If you are involved with managing test automation and need to understand the key issues in making test automation successful, join Mark for this enlightening tutorial.More Information
As testers, we know that we can define many more test cases than we will ever have time to design, execute, and report. The key problem in testing is choosing a small, “smart” subset from the almost infinite number of tests available that will find a large percentage of the defects. Join Lee Copeland to discover how to design test cases using formal black-box techniques, including equivalence class testing, boundary value testing, decision tables, and state-transition diagrams. Explore examples of each of these techniques in action. Don’t just pick test cases at random. Rather, learn to selectively choose a set of test cases that maximizes your effectiveness and efficiency to find more defects in less time. Then, learn how to use the test results to evaluate the quality of both your products and your testing. Discover the test design techniques that will make your testing more productive.More Information
The nature of exploration, coupled with the ability of testers to rapidly apply their skills and experience, make exploratory testing a widely used test approach—especially when time is short. Unfortunately, exploratory testing often is dismissed by project managers who assume that it is not reproducible, measurable, or accountable. If you have these concerns, you may find a solution in a technique called session-based test management (SBTM), developed by Jon Bach and his brother James to specifically address these issues. In SBTM, testers are assigned areas of a product to explore, and testing is time boxed in “sessions” that have mission statements called “charters” to create a meaningful and countable unit of work. Jon discusses—and you practice—the skills of exploration using the SBTM approach. He demonstrates a freely available, open source tool to help manage your exploration and prepares you to implement SBTM in your test organization.More Information
In many organizations, agile development processes are driving the pursuit of faster software releases, which has spawned a set of new practices called DevOps. DevOps stresses communications and integration between development and operations, including continuous integration, continuous delivery, and rapid deployments. Because DevOps practices require confidence that changes made to the code base will function as expected. automated testing is an essential ingredient Join Jeff Payne as he discusses the unique challenges associated with integrating automated testing into continuous integration/continuous delivery (CI/CD) environments. Learn the internals of how CI/CD works, appropriate tooling, and test integration points. Find out howpto integrate your existing test automation frameworks into a DevOps environment and leave with roadmap for integrating test automation with continuous integration and delivery.More Information
The key to successful testing is effective and timely planning. Rick Craig introduces proven test planning methods and techniques, including the Master Test Plan and level-specific test plans for acceptance, system, integration, and unit testing. Rick explains how to customize an IEEE-829-style test plan and test summary report to fit your organization’s needs. Learn how to manage test activities, estimate test efforts, and achieve buy-in. Discover a practical risk analysis technique to prioritize your testing and become more effective with limited resources. Rick offers test measurement and reporting recommendations for monitoring the testing process. Discover new methods and develop renewed energy for taking your organization’s test management to the next level.More Information
Cloud computing is rapidly changing the way systems are developed, tested, and deployed. New system hosting capabilities—software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS)—are forcing us to review and revise our testing processes. At the same time, cloud computing is affording us opportunities to employ new test tooling solutions, which we call testing as a service (TaaS). In this technical session, Martin Pol and Jeroen Mengerink focus on testing SaaS systems, describing relevant IaaS and PaaS capabilities along the way. They discuss how to test performance of the cloud itself and ways to take advantage of the resource elasticity afforded by cloud computing. Martin and Jeroen explore the risks―some traditional, others completely new—that arise when organizations implement cloud computing and describe the tests you need to design to mitigate these risks.
Delegates will receive a free copy of the book Testing Cloud Services by Kees Blokland, Jeroen Mengerink, and Martin Pol.More Information
In automation, we often use several different tools that are not well integrated. These tools have been developed or acquired over time with little consideration of an overall plan or architecture, and without considering the need for integration. As a result, both efficiency and effectiveness suffer, and additional time and money are spent. Ensuring that tools we currently have, or the tools we develop or acquire in the future, work well with other application lifecycle tools is critical to our testing team’s success. We must drive the adoption of automation across multiple project teams and departments and communicate the benefits to our stakeholders. Join Mike Sowers as he shares his experiences in creating an automation plan, developing an automation architecture, and establishing tool metrics in multiple organizations. Mike will discuss both the good (engaging the technical architecture team) and bad (too much isolation between test automators and test designers) on his automation journey in a large enterprise.More Information
Running a test project can be a challenge. Running a number of test projects as part of a portfolio can be even more challenging. However, most challenging of all can be running a group of projects in which every project needs to merge at a single end point. Geoff Horne considers: How does a program test manager (PTM) slice up the testing work packages and then group them by “like” types into discrete projects? How does the PTM determine the best approach for each project while maintaining the most advantageous approach for the overall program? How does each project fit into the overall test strategy? These and other questions are the everyday challenges of the PTM. Maintaining forward momentum at the required rate across many different tracks, all heading for a single end point, requires skill and experience at many levels. Join Geoff to learn how to qualify, quantify, and effectively run any size test program like a well-oiled machine.More Information
It’s not enough to verify that software conforms to requirements by passing established acceptance tests. Successful software products engage, entertain, and support the users' experience. Goals vary from project to project, but no matter how robust and reliable your software is, if your users do not embrace it, business can slip from your hands. Rob Sabourin shares how to elicit effective usability requirements with techniques such as storyboarding and task analysis. Together, testers, programmers, and users collaborate to blend the requirements, design, and test cycles into a tight feedback loop. Learn how to select a subset of system functions to test with a small group of users to get high value information at low cost. Learn how usability testers can take advantage of naïve questions from novice users as well as the tunnel vision and bias of domain experts. Rob shares examples of usability testing for a variety of technologies including mobile and web-based products.More Information
Many software systems are required to process huge combinations of input data, all of which deserve to be tested. Since we rarely have time to create and execute test cases for all combinations, the fundamental problem in testing is how to choose a reasonably-sized subset that will find a large percentage of the defects and can be performed within the limited time and budget available. Pairwise testing, the most effective test design technique to deal with this problem, is unfortunately not understood by many testers. The answer is not to attempt to test all combinations of all values for all input variables but to test all pairs of variables. This significantly reduces the number of tests that must be created and run but still finds a large percentage of defects. With examples of the effectiveness of pairwise testing, Lee Copeland demonstrates this technique through the use of orthogonal arrays, James Bach’s all-pairs algorithm, and Microsoft’s PICT tool. Learn to apply the pairwise testing technique as you work through a number of hands-on exercises.More Information
Today’s software applications are often security critical, making security testing an essential part of a software quality program. Unfortunately, most testers have not been taught how to effectively test the security of the software applications they validate. Join Jeff Payne as he shares what you need to know to integrate effective security testing into your everyday software testing activities. Learn how software vulnerabilities are introduced into code and exploited by hackers. Discover how to define and validate security requirements. Explore effective test techniques for assuring that common security features are tested. Learn about the most common security vulnerabilities and how to identify key security risks within applications and to use testing to mitigate them. Understand how to security test applications—both web- and GUI-based—during the software development process. Review examples of how common security testing tools work and assist the security testing process. Take home valuable tools and techniques for effectively testing the security of your applications going forward.More Information
You have just been assigned a new testing project. Where do you start? How do you develop a plan and begin testing? How will you report on your progress? Paul Holland shares new test project approaches that enable you to plan, test, and report effectively. Paul demonstrates ideas, based on the Heuristic Software Test Model from Rapid Software Testing, that can be directly applied or adapted to your environment. In this hands-on tutorial, you’ll be given a product to test. Start by creating three raw lists (Product Coverage Outline, Potential Risks, and Test Ideas) that help ensure comprehensive testing. Use these lists to create an initial set of test charters. We employ “advanced” test management tools (Excel and whiteboards with Sticky Notes) to create useful test reports without using “bad metrics” (counts of pass/fail test cases, % of test cases executed vs. plan). Look forward to your next testing project with these new ideas and your improved planning, testing, and reporting skills.More Information
Data warehouses have become a popular mechanism for collecting, organizing, and making information readily available for strategic decision making. The ability to review historical trends and monitor near real-time operational data has become a key competitive advantage for many organizations. Yet the methods for assuring the quality of these valuable assets are quite different from those of transactional systems. Ensuring that the appropriate testing is performed is a major challenge for many enterprises. Geoff Horne has led a number of data warehouse testing projects in both the telecommunications and ERP sectors. Join Geoff as he shares his approaches and experiences, focusing on the key “uniques” of data warehouse testing including methods for assuring data completeness, monitoring data transformations, and measuring quality. He also explores the opportunities for test automation as part of the data warehouse process, describing how it can be harnessed to streamline and minimize overhead.More Information
Communication is at the heart of our profession. No matter how advanced our testing capabilities are, if we can’t convey our concerns in ways that connect with key members of the project team, our contribution is likely to be ignored. Because we act solely in an advisory capacity, rather than being in command, our power to exert influence is almost entirely based on our communication skills. With people suffering information overload and deluged with emails, it is more important than ever that we craft succinct and effective messages, using a range of communication modalities. Join Thomas McCoy as he draws on techniques from journalism, public relations, professional writing, psychology, and marketing to help you get your message across. Key themes include: non-verbal communication, presentation skills, persuasive writing, influencing skills, graphic communication, and communicating in teams and meetings. We will use a range of hands-on exercises to practice the concepts being discussed.More Information
In both agile and traditional projects, keyword-driven testing—when done correctly—has proven to be a powerful way to attain a high level of automation. Many testing organizations use keyword-driven testing but aren't realizing the full benefits of scalability and maintainability that are essential to keep up with the demands of testing today's software. Hans Buwalda describes the keyword approach, and how you use it to can meet the very aggressive goal that he calls the "5 percent challenge"―automate 95 percent of your tests with no more than 5 percent of your total testing effort. Hans also discusses how the keyword approach relates to other automation techniques like scripting and data-driven testing, and the ways keywords can be used for specific situations like graphics, multimedia, and mobile. Use the information and the real-world examples that Hans presents to attain a very high level of maintainable automation with the lowest possible effort.More Information
Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The key to good estimates is to understand the primary variables, compare them to known standards, and normalize the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when very little is known about the project and what is known is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking and estimation can become a nightmare. Rob Sabourin provides a foundation for anyone who must estimate software testing work effort. Learn about the test team’s and tester’s roles in estimation and measurement, and how to estimate in the face of uncertainty. Analysts, developers, leads, test managers, testers, and QA personnel can all benefit from this tutorial.More Information
With the adoption of agile practices in many organizations, the test automation landscape has changed. Bob Galen explores current disruptors to traditional automation strategies, and discusses relevant and current adjustments you need to make when developing your automation business case. Open source tools are becoming incredibly viable and beat their commercial equivalents in many ways―not only in cost, but also in functionality, creativity, evolutionary speed, and developer acceptance. Agile methods have fundamentally challenged our traditional automation strategies. Now we must keep up with incremental and emergent systems and architectures and their high rates of change. Bob explores new automation strategies, examining strategies for both greenfield applications and those pesky legacy projects. Learn how to wrap a business case and communication plan around them so you get the support you need. Leave the workshop with a serious game-plan for delivering on the promise of agile test automation.More Information