3 - 5 minute read

The Endless Load Test

The Endless Load Test

Or, why you need to include load testing in your CI/CD pipeline

The Endless Load Test, it sounds like the title of a horror movie no one ever asked for. Probably a better way to look at this topic is: Load Testing While You Sleep. It’s a well known fact that the earlier you detect a defect that hurts performance, throughput and/or stability, the easier it is to mitigate. In my experience, 70% to 80% of the problems I’ve helped uncover during load testing are reproducible at much lower scales. It is much less painful (and less expensive) to deal with such bugs testing with 20-500 virtual users than during your seasonal load testing which may use tens to hundreds of thousands of virtual users.

You’re already performing functional tests as part of your release cycles (right?), so taking those a step further to load testing is easier than you think. There are just a few things you need to keep in mind:

Keep it to scale

Keep testing in a scaled environment

Whether you’re using traditional system specifications like CPU cores and gigabytes of RAM, or cloud-native services such as Lambda, Aurora, containers as a service etc, keep your testing environment scaled. Are you running 16 application servers with a 32 core database? Then you could aim for 2 app servers with a 4 core database for example. Using cloud-native services? Then make sure you scale capacity, throughput reservation and so forth proportionally in your development environment.

Reuse test scripts as much as possible

Reuse test scripts

Remember that automated functional testing you’re already doing? Look for load testing tools or services that are based on the same technology. Doing functional testing with JMeter? Then use JMeter for your load testing as well. Using real browsers from Selenium scripts? There are load test services and opensource tools for that. One caveat to remember is that most large scale load testing uses protocol emulation rather than real browsers for resource/cost reasons. While it’s admittedly a pain to maintain 2 flavors of scripts, it may be justified by the cost savings.

Define your KPI’s, then actually use them

Define performance KPIs

KPI’s for load tests can be more nuanced than for a functional test, where the results are either it worked, or it didn’t. How do you know if your load tests “worked”? Try starting with your production environment as a baseline, think about KPI’s based on:

  • Throughput such as requests per minute
  • Loading times for pages, click-actions and even background actions that are on the critical path (this is an area where real browser tests have an advantage having real javascript engines etc as everything is automatically included in the test)
  • Don’t forget to take into account or filter out as appropriate, requests to external services and third parties

Last but definitely not least, monitor your results! If your load test results are heading south in your releases, you don’t want to miss the warning signs. This may require automated alerting based on the KPI’s you’ve defined. The last thing you want is to have these results disappear into the the Inbox black hole where so many SLA reports have gone.

This article is published before on LinkedIn.

Facebook
LinkedIn
Twitter

Place a suiting CTA right here

Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, link within text 

This website uses cookies

With these cookies, we and third parties can collect information about you and your internet behavior, both within and outside our website. Based on this, we and third parties tailor the website, our communications, and advertisements to your interests and profile. You can find more information in our cookie statement.

Accept Reject More options

This website uses cookies

With these cookies, we and third parties can collect information about you and your online behavior, both within and outside our website. Based on this, we and third parties tailor the website, our communications, and advertisements to your interests and profile. You can find more information in our cookie statement.

Functional cookies
Arrow down

Functional cookies are essential for the proper functioning of our website. They allow us to enable basic functions such as page navigation and access to secure areas. These cookies do not collect personal information and cannot be disabled.

Analytical cookies
Arrow down

Analytical cookies help us gain insight into how visitors use our website. We collect anonymized data about page interactions and navigation, enabling us to continuously improve our site.

Marketing cookies
Arrow down

Marketing cookies are used to track visitors when they visit different websites. The goal is to display relevant advertisements to the individual user. By allowing these cookies, you help us show you relevant content and offers.

Accept all Save

Subscribe to our newsletter!

Subscribe to our newsletter!