STARWEST 2022 Concurrent Session : A Performance Testing Pyramid


Thursday, October 6, 2022 - 1:30pm to 2:30pm

A Performance Testing Pyramid

When we code a new class method, does its execution time fit within the product’s SLA for response time? When we provide an API for a real-time system, does it meet its performance budget? When we’re managing a data warehouse system, what is the latency between the ingestion component and the persistence server? When we’re planning to deploy a large-scale web application, how many servers do we need to support 1,000 requests per second and respond in 3 seconds? We’ve learned that the need for performance data spans from requirements engineering through system deployment: when code is written, components are integrated, and systems are deployed. We’ve also learned this requires a holistic approach of well-designed layers of tests to both identify issues quickly and gain confidence in our system as a whole. The performance testing pyramid is intended to be parallel to the functional testing pyramid, and measures performance strategically with micro-performance tests at unit and integration levels, and macro-performance tests at the system level. We’ll discuss our performance testing pyramid, its benefits, the trade-offs, and how such a portfolio of micro-performance and macro-performance tests may be targeted to meet demand for continuous performance data.
Trimble PPM, Inc.

James Wood is Software QA Architect for Trimble PPM. As researcher, developer, and tester, he has worked 36 years for firms including Siemens, Google, and Ultimate Software. He holds B.S. and M.S. degrees from Carnegie-Mellon University and the University of Pennsylvania. Robert Vanderwall, quality advocate, is Data Scientist for Trimble PPM. He has worked 40 years in hardware, software, quality assurance, process, and data engineering for companies ranging from a 4-person startup to AT&T Bell Labs. He holds several patents and earned his Ph.D. at Case-Western Reserve University. Katie Fox is Senior QA Engineer for Trimble PPM. She has worked 9 years as a tester, at Trimble since 2021 and Ultimate Software prior. She earned her B.S. Computer Science degree from the University of Central Florida.