Load testing allows you to gather metrics on the performance and behavior of your application under normal or anticipated peak load conditions. This not only helps you make educated decisions about your application and environment but provides a level of confidence your users should not experience a degradation in performance or worse an outage. By being proactive and making course corrections when issues are encountered during tests.
It involves putting sufficient load on an application while it remains up and running so you are able to gather metrics about the applications performance and the environment. These metrics should include things like throughput-rates, response times, memory consumption and CPU utilization.
Testing Strategies
There are a number of different types of tests strategies that can be incorporated into your strategy. The following strategies, however, provides more than sufficient insight into how your application behaves under load and will help identify any potential issues that may occur.
Baseline Testing – is used to determine how your application and environment respond under normal traffic conditions. Once a baseline is established you can run subsequent tests against each new release or when a change is made to the environment. This allows you to determine if anything has been introduced to adversely affecting performance. The opposite may also be true to improving performance and in the process establishing a new baseline.
Stress Testing – is designed to take your application to almost breaking point. Starting with your baseline test increment the load until your application is at a breaking point. This gives you some insights into the capabilities of your application and helps you make informed decisions when scaling your environment.
Soak Testing – involves running your baseline test for much longer periods of time for example 24-72 hours. This type of test will help tease out any potential issues that might occur with the application when running over longer periods of time, for example, memory leaks, etc.
Determining your Baseline
In order to create your baseline, you need to know which test scenarios you are going to run and how many virtual users should be used to run the test scenarios.
Deciding on the test scenarios is fairly straightforward. These are going to be the most common or expected journey a user will take on your site and adequately tests the main functional areas. If for example, you have an e-commerce application they will probably search for products, add one or more to the cart, check out and finally the confirmation. They might even create an account or look at some other content pages like a returns policy etc.
How many virtual users/concurrent users should be used to run these scenarios for your baseline load test? Hmm, this one requires a bit more thought and you’ll need to consult the business. It’s important that you understand the difference between Visitors and Concurrent users. Concurrent meaning how many of those visitors are actively making requests at the same time. If you have access to Google Analytics you should have all the information you need to work this out using the following formula:
Users = (Hourly Sessions x average session duration in seconds) / 3600.
For example, if peek hourly traffic on a site is 6000 visitors and they spend on average 4 minutes then the number of concurrent users = (6000 x (4* 60)) / 3600 = 400
When analyzing your traffic be sure to take into consideration peak traffic times and be careful if you are thinking of using averages over a period of time as these could be considerably lower than the peak traffic times. I also recommend you increase the calculated value by 5-10% to use with your baseline. This ensures your baseline tests adequately test your application and gives confidence the application is more than capable of handling peak traffic periods.
Running load tests
There are many tools on the market you can use help you create and run your load tests. From cloud base to standalone and distributed applications.
I’m a JMeter fan having been introduced to its capabilities many years ago and seeing how it had helped load test some of the worlds busiest travel websites. JMeter is a Java application and there is no shortage of useful information to help you get started. Best of all it is free.
It was designed to be used to simulate load and scenarios to help with performance testing of your web applications. While you probably could use it for functional testing it is not what it was built for and I recommend using other tools like Selenium for that purpose.
Over a series of posts, I’ll explore load testing with JMeter covering basic to more advanced topics to help you load test your Sitecore application:
- Load Testing made easy with JMeter
- Load Testing with JMeter Advanced
- Capture & Monitor JMeter metrics in real-time with InfluxDB & Grafana
Neil Killen 
2 thoughts on “Measuring performance under load.”