This is a continuing theme of mine and I will undoubtedly write more on it.  For the faint-hearted where math(s) is concerned - look away now - and come back in the paragraph that starts "If anyone is still with me...".

Statistics works on "confidence levels".  If I have 5000 people whose opinion I would like to know and I ask 1000 of them a few questions, how confident can I be that I have the correct answer?  The answer is you can be 95% confident with a 2.8% error level.  I know, its horrible but that is the way it works.

So, what has this got to do with 360 degree appraisal?  Well, let's say you have 15 people working for you and you can't help yourself but want to know a score for your ability to "Delegate effectively".  Typically, we would ask, say, 6 direct reports to give feedback.  If they all give feedback then you the error level you would now have on your 95% confidence rate is 31%.  That is, it is a wholly unreliable number.

But, if you read most 360 reports you will see scores of 4.23 on Delegate effectively as if there is some amazing precision in the responses.  There just plain isn't.  Add in all of the issues around rater selection, halo effects, and getting good questions and the chance of needing two decimal points is remote!

If anyone is still with me - and I welcome those who skipped the above - 360 should be used to aggregate responses not average them.  A simple report that lets you know how people answered each question has all of the information you require.  There is some merit in getting averages over a group of people to determine trends - indeed we use this widely ourselves.

Here is one that we use



I would argue that this tells you everything you need to know.  (Note, this person had 4 managers!).

360 degree appraisal is very powerful - people read a lot into their own 360 feedback.  Statistics can be useful but we need to be careful we don't overuse them.

Brendan