Performance testing results produce lots and lots of figures, statistics and graphs. To the uninitiated stakeholder with lots of other tasks to balance it looks like pure hell, particularly if they are asked to sign off on the basis of these.
Performance testers tend to underestimate this stage in terms of time and importance, quite often I’ve seen ‘out of the box’ graphs, figures and percentiles presented at the end of performance test cycle with no summary. These technical and ‘out of the box’ figures should be used as evidence to support a management summary. I believe it is the duty of the performance role to present and interpret these figures into the context of the business. If the performance requirements & tests have been well defined upfront this should be straightforward. The businesses essentially want a GO/NO GO decision with any associated risks highlighted. Stakeholders are busy – they do not have the time or inclination to pour over the figures and interpret them themselves – they shouldn’t have to.
Taking the results and providing an interpretation can take an entire day – I find its worth the time and effort. This is also a great opportunity to try and get any stakeholders asking questions and aiding their understanding. One of my ‘quiet’ measures of the quality of the summary I produce is the number of responses I receive – If there are questions then I’m happy. Generally it means some of the stakeholders have taken time to read it and feel comfortable asking questions. Trying to engage the stakeholders and generate interest means they can begin to understand the work that is being done – it also gives clearer visibility on the dark art of performance testing that has been black boxed to the outside world for X weeks. If I don’t receive any questions then I begin to worry – they haven’t understood or aren’t interested.
- Interpret the technical statistics – filter the unnecessary and highlight the interesting
- Provide a Summary that is management based – highlight risks and areas of further investigation
- Give a Go/No-Go indication
- Attempt to engage and aid understanding of the results you present
- No questions or interest should be a red flag
Take time to create a report everyone can understand, organise a meeting to present the results and send out the summary at least 24 hours in advance. Give people a chance to look at and digest the results – so they can ask sensible questions in the presentation. Only by attempting to engage can we aid understanding.
Questions/Statements I’ve come across:
Q. “I’ve got this result which looks slightly slower, I haven’t got the authority to sign off Performance based on this result”
A. Highlight this result in the management summary and get the associated stakeholder to sign off on the result and risk prior to go-live.
S. “Its not my job to provide an interpretation, I am just here to performance test and present the results“
A. You are the domain expert; your role includes providing an interpretation of the statistic results into the context of the business. Stakeholders haven’t got the time to try and interpret statistics, this is part of your role. Stakeholders are also looking to your role to provide an indication of performance quality – if you cant, who can?