Forewarned : this is a topic that bothers me more than it should. I was reading an article on team 360 degree surveys. Very interesting – an area we are looking at. I then followed the link in the article to the company that was offering the service. I looked at the report the software produced and the first table was a set of percentage “scores” for self compared to others.
You just can’t use percentages like that!
Let’s see if I can explain without becoming too nerdy.
On one competency, the “self” will have rated themselves, let’s say 4 times. Let’s assume the rating scale was Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree. Typically we get asked to score this as 1, 2, 3, 4, 5.
So, if the self scores the 4 behaviours on a particular competency as Agree twice, and Strongly Agree twice then you sometimes see a “score” of 4.5. Now, that is bad practice in itself – for reasons I’ll explain another time. But to say they scored 90% is shocking.
Percentage is scoring out of 100. If I score 943 out of 1000 shots, then 94.3% is accurate because it is fair to say that on balance you will get 94 and a bit goals every 100 shots. But if I shoot 4 times and score 3 of them, it might be mathematically 75% but none of us would feel confident that if I shot another 96 times I’d score 72 more goals. I am reminded of the mathematical oddity of saying that Jesus was betrayed by 8% of his disciples. It just isn’t what you are supposed to use percentages for and they don’t work very well when you do.
So, what is a reasonable use of percentages.
Well, over the whole of a 360 you can reasonably say something like “The direct reports rated ‘strongly agree’ 40% of the time”. When Bowland Solutions suggests its 360 degree appraisal reports it often recommends a table that reflects the percentage frequency of responses from each group as a handy summary.