So there is a new kid on the performance testing block. SOASTA CloudTest – a Load Testing tool, which unsurprisingly is hosted in the Cloud. I’ve recently had time to get hands on with the tool and thought I would attempt to sum up the the main benefits here:
Free Demo Version: There is no license cost to get up and running – Download Cloudtest lite for free and run up to 100 concurrent virtual users. This version is as close as you can get to a full production grade version of any other tool I have seen. The only real restriction is the number of users – if you want to go beyond that then you simply upgrade to the Pro version.
Installation: No need for the associated headaches of installation, managing injectors, controllers, analysis tools etc. It all lives in the Cloud and its taken care of for you. The software also has advanced deploy and diagnostic capabilities – if a service fails on a cloud host, it detects and redeploys automatically.
Upgrade: The Free and Pro versions now come with an ‘update’ button. Click on this and the version is updated to the latest. This is fantastic. I’ve spent many hours upgrading different components of loadtesting tools, backing up projects and then crossing my fingers when re-executing tests. How many support call responses start with the ‘Please upgrade to the latest version…’. Those days are gone. Press the button and its done… Its the small things like this that make a big difference, it save hours of time and hassle… and makes our lives a lot easier.
Hardware Cost: As well as saving the associated cost and setup for hardware – you pay only for the server hours you need when running large tests. So if you need to test for 20,000, 100,000 … or 1million users then you only pay for the hardware as and when it is being used.
So thats the standard benefits of the Cloud SaaS that we have come to know and expect. If we put any other tool in the Cloud we would expect the same right? So I’ll dive into the more specific benefits of the actual SOASTA load testing tool.
Scripting: The tool has a professional front facing GUI driven scripting engine. I’ve never seen such a rich visual interface living inside a browser. Be aware though, if you are a professional performance tester its a very different approach. Its like relearning how to eat but with chopsticks. More on this in a later article. If you are familiar with a LR type approach then you will find the scripting approach frustrating. Unfortunately the scripting approach (for me) is one of the areas where this tool is weakest.
Injectors genf20: Not only is the maintenance of injectors seamless, the tool allows cherry picking of load injection from various locations around the world. E.g. I can choose to inject 40% of my users from the East Coast of America, 20% from the West cost and 40% from the UK. SOASTA have a large number of geographically distributed locations to choose from from – all at the touch of a button.
Real Time Load Analysis: Performance testers know that they are flying blind without the ability to view what is going on underneath the bonnet. Every conceivable graph is made available in realtime. You can overlay graphs with one another, create your own dashboards on the fly and drill down into any areas that look problematic. SOASTA have invested heavily in creating in-memory objects that make ALL data available at runtime. This allows detection and investigation of issues as they happen. This means I no longer have to wait for a performance test to finish, analyse the results – spot a potential issue, raise it (lengthy discussions entails) and then re-run to prove. I can pick up the phone and ask someone to look immediately. Here are some screen shots of the analysis graphs [Insert Link here] – they look nice to.
Monitors: Proactive agents and passive monitors are available at no additional cost. Hooks are provided for all major APM’s.
Uncomplicated Pricing Structures: Pricing is mainly focused around the number of load injector machines you require per hour. There are other pricing structures for professional services – but if you wish to take control and do this yourself its very straightforward.
Protocol Adaptors: Flash, Web Services and HTTP are supported, fairly limited. SOASTA have decided the way of the world is the web.
So anyone can now get a fully featured, enterprise version of a load testing tool – without the associated costs of maintenance, hardware and upgrade cycles. SOASTA have different pricing structures depending on your needs. The main structure is geared around the flexible number of load testing hours required (e.g. 2k, 20k,1 million users for X Hours) i.e. You are charged only for what you use – ‘on demand’.
I foresee a major uptake here for Professional Performance Testers and IT service companies. They can quickly develop using the free version and then inject high load and help diagnose for customers when required with the Pro version. End users are no longer forced into a complicated licensing or buying agreements – they only need to pay for the load testing hours required (and the consultancy).
The simplicity of the pricing, maintenance, cost of ownership, finished quality and features of the product will be compelling for many.