Performance Test Consultant

I was recently asked a series of questions by a site and thought the answers would give a more in depth view of myself, here are the questions and answers:

Q.What is your technical background?

A. I learnt to program when I was 13 – it was a vic20, I then swiftly moved onto 6502 for the BBC micro. Uni studying computer science was an obvious and easy progression for me. First job was writing rendering engines (C++) for virtual reality simulators. I would describe myself as a techie at heart – I’m genuinely interested in how things work. I think having a strong and long background in IT enables me to grasp new concepts easily – which is great when I have to go into different companies and need to understand their systems quickly. I’ve a 2:1 in Computer Science, ISEB Practitioner and SCRUM Master Certified.

Q.Do you consider yourself more of a software developer or QA professional?

A. We should get out of the habit of separating them so readily. I feel strongly that every software developer should QA their work. Its not good enough to code and then relinquish QA to another team. Its lazy and it increases delivery time, wastes effort and increases cost. Everyone should be a QA professional within their own field. I think a large dedicated QA team is a good measure to the inefficiency of an IT project. I’m going to write a post on this very topic.

Q. How much involvement do you have with load and performance testing?

A. Lots. I’ve tested mission critical and highly transactional systems. e.g. Trading platforms with lots and lots of money moving through them. I wish I could figure out a way of just getting a small percentage of that, a bit like superman 4.

Q. What would you say is the difference between load testing and performance testing?

A. Load testing is testing with anticipated load, Performance testing is a generic umbrella term for all types of testing e.g. Soak, Load, Volume, Spike …. I’ve written this article on my blog: performance-testing-definitions

Q. What do you think is the most important aspect of load testing?

A. Understanding the requirements and getting the workload model correct. Your performance test is only as accurate as the model you are simulating.

Q. Who are the most influential leaders in the field of performance engineering?

A. At a deep level I believe the Facebook engineers are, they get it. They scale their software better than anyone else – php not working quickly enough – lets write HipHop, its increases speed, reduce datacenter space & cost. They spearheaded pagelets (Bigpipe). They measure, inspect and then improve the software if possible (True Continuous Improvement) – obvious, but I’ve seen a lot of companies not do this because they don’t understand their own technology.

Q. How do you determine the load to apply to the target app during load testing?

A. Talk to all parties, not just one and look to evidence the requirements.

Q. Do you prefer using requests per second (RPS), concurrent users, or some other metric to define load?

A. Whatever make sense in the customers context. I always think of defining the load in a way the customers will understand. You get better feedback and more accurate requirements as a result.

Q. What is the biggest change you have witnessed in the way people conduct load testing?

A. There is a quiet move away from Loadrunner and its going to become an avalanche, its been underdeveloped and overpriced for a long time.

Q. Do you feel like performance testing is an accepted critical part of the development life cycle?

A. Unfortunately in the majority of cases it only becomes part of the cycle after something horrible has happened. Then it becomes a critical part. Thats human nature in general.

Q. What are the key KPIs you track for performance testing and tuning?

A. Transaction response times, Memory, CPU, transaction failure graphs are the ones I cannot live without. Graphs can tell you a whole story simple metrics cannot.

Q. Do you have any cool tips/tricks for performance tuning?

A. Don’t try and replicate in software what the DB can do. I’ve seen a lot of companies try and re-invent the wheel and take out logic from the DB (such as caching) and put in their own solutions. Then they find they get into all sorts of trouble when the item in the cache is updated from an external source.  They essentially start replicating the logic of the DB. Always look to tune and improve the DB – there are 1000’s of man hours invested in that technology. Your problem will not be unique and will have been solved.  Part of the problem I’m now seeing is that software engineers are sometimes just focused on the narrow field of software engineering and don’t appreciate the holistic system.  Instead of getting in a DB tuning expert they may (for example) just take out items and put them in a cache – this increases the code complexity, man hours and cost.  See also this article about being proactive about performance – visibility and monitoring!!

Q. How does cloud computing affect the future of automated testing?

A. It reduces the cost of ownership, hardware and makes it easier for teams to be off-shore. Its a slow revolution and its for the better.

Q. How important is web performance to your clients?

A. Critical – more and more people are doing business over the web. The competition is high and people can switch easily. Customers expectations are also extremely high. Waiting longer than 3 seconds is now deemed unacceptably slow when navigating through sites.

Q. Please share with us an interesting story about your experience with software development or load testing.

A. I’ve seen some pretty silly things. One of the client sites I was called into had a live performance issue – its affected their whole business. They bought a consultancy in to estimate where they thought the capacity was and then decided to re-engineer the whole architecture at a hugh cost. All without physically proving this first – nuts. By the time they had got new hardware in to performance test their existing architecture they discovered it could take them through the next two years of growth – easily. And this was without no performance tuning – what a waste.  The sad thing is I’ve seen far more mad things than this.

Q. Please provide a link to your favorite blog post that you have written and tell us what you like about that one.

A. At the moment this one: choosing-the-correct-load-testing-tool I’ve been thinking hard about how I could easily split all the load testing tools out there and explain to someone coming at it from a fresh perspective.

Q. What are your favorite testing or development blogs (other than your own)?

A. At the moment I’ve come across this http://bettertesting.co.uk/. Its very refreshing, informative and the I like the fact that Darren is well rounded.

Q. If you could make a career from one of your favorite hobbies, what would it be?

A.  Personal training instructor – I enjoy teaching people how lose weight, find out what motivates them and work out how to change their body through exercise they will enjoy

See also What Makes A Good Performance Tester

Leave a Reply

Your email address will not be published. Required fields are marked *