How this links to 360 degree feedback will follow!
Let’s assume we have a basic traffic light system. And we find out that we have the following distribution
- Red 50% of the time
- Amber 10% of the time
- Green 40% of the time
And someone wants to know what colour it is on average. What to do?
Average requires numbers.
So, let’s give Red the number 1, Amber the number 2, and Green the number 3. A bit of Maths will find the average now (50% *1) + (10% * 2) + (40% *3) … 1.9 is the answer.
So our average is 1.9 which is nearest to Amber (which we gave the number 2). So on average the colour of the traffic light is Amber … somewhere in the middle.
We know this is wrong – the light is on amber least of all – but it was an attractive solution somehow.
Even more tempting is to ask people to respond to a question with responses that can be
- Strongly Agree
- Neither agree nor disagree
- Strongly Disagree
and then give people a score out of 5 for how people answered on average.
Let’s say we have done that and the average is 3.1. What does that mean? We’re going to say that on average the respondents roughly "neither agree nor disagree" with the statement. But go back to our traffic light example… the colour that the traffic light is least often is amber, what’s to say the same hasn’t happened here?
Statisticians will tell you that the underlying problem here is that you are treating categorical data as if it is numerical data; if you are very unlucky, you may find yourself in a debate about likert scales and the Polytomouse Rasch model. What I want to highlight is, this superficially simple concept of giving each point a number and averaging the responses is not that simple and may lead to inaccurate conclusions.
Way’s round it? Either don’t use averages in your report, or make it clear that you are using the scale to reflect a mark out of 5. That at least gives your average credibility even if it doesn’t get around the problem of the average obscuring the underlying scoring.
Note, this doesn’t stop you using numbers or charts in your report – I’ll discuss that in another post.
This article forms part of the structure of a new white paper I am writing on reporting in 360 degree appraisals (I promise this is the heavily statistical bit and the rest is more down to what we see as best practice!). If you sign up to our white papers, then you will receive that document as it is completed. Click here to sign up.