March 13, 2019
As you come close to launching your new web application, whether it be Sitecore, Node or plain ol’ HTML, it’s always good to validate how well it performs.
The cloud opens up lots of possibilities for how to approach this – including lots of online LTAAS (er, is load test as a service even a thing!?!? :))
Iteration 1 – LTAAS with Azure Devops
We are using Azure Devops within our current project, so thought it be good to give their load testing features a blast. This came with mixed success, and a mixed $$$ cost.
Pro’s
- You don’t need to manage any of the kit
- The sky’s the limit with the amount of concurrent machines to run (< 25)
- It supports various methods for building a script
Con’s
- The feedback loop can feel slow
- You get limited support for JMeter scripts, and limited graph’s of your results. Note, this could be due to inexperience with the tool
- It costs per minute of load test you run. We managed to un-wittingly rack up quite a substantial bill with a misconfigured script.
Iteration 2 – DIY
Another approach is that you actually setup the infrastructure yourself. For our capacity and requirements this ended up being a much more favourable option – once we’d managed to get the most out of our kit.
Pro’s
- Assuming you use JMeter, you can quickly iterate through tests and get a wide spread of results as you go
- If you need more grunt, you can always increase the box spec’s
Con’s
- You need to tune the box to get the most out of it
- Large boxes in e.g. AWS cost $$$
Configuring things yourself
Here are a few steps to follow if you really want to max out your load test box, as well as you web infrastructure:
- Pick a box with plenty of RAM – we opted for and AWS
r5.2xlarge – 8 core and 64GB RAM - Ensure JMeter can use all the RAM it can. Within JMeter.bat you can set the heap size available to the program – by default this is 512mb. If you add
set HEAP=-Xms256m -Xmx60gthen JMeter will sap up all 60GB of RAM it can - Ensure Windows can use as many TCPIP connections as possible. Again, by default this is quite low. You need to set 2 registry keys – see http://docs.testplant.com/epp/9.0.0/ePP/advovercoming_tcpip_connection_li.htm for more details.
- Until we’d set these values, our tests would bomb out after a couple minutes as the box simply couldn’t connect to our website any more.
Other tips
JMeter has some really good plugins for modelling load, in particular around step’d load and realtime visualization of results.
I’d recommend checking out:
- https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/
- https://jmeter-plugins.org/wiki/TransactionsPerSecond/
Some good additional reading
Original article was posted on Boro2g Blog: Build yourself a JMeter load testing server
