I've written before on how to choose an online 360 degree feedback system and you can't help but feeling a bit of a fraud when you are part of an organisation that provides them.  The flipside is that to write about implementing 360 degree feedback without talking about systems is nonsense.  So, here goes:

Choosing an on-line system is much more about ensuring that the system serves you rather than the other way around, than it is working to a checklist.

You are looking for a system that is geared to meet your requirements rather than forcing a way of working upon you.

Here are the principles we would look for.

The system should :

• make it simple
• ensure the only time spent by people is on completing or reviewing feedback rather than clicking all over the place
• make error checking easy
• remove administration tasks / make them easier
• improve reporting and make it more timely
• There should be no need at all for training in the use of the system – a 360 system should be simple, clear, and obvious.

Get references.  I know that most suppliers are going to pick their favourite clients but ask for the last 2-3 projects and that is then harder.

Brendan

The rating scale in 360 degree feedback can be oddly contentious.  Everyone has an opinion.  Here is our view based on a lot of experience (which of course makes us right !)

How many rating points?

We recommend as few as possible.
Keep it simple. Our standard scale would be 3 points plus 'not applicable'. So; Strength, Competent, Development Area would be a great scale. We work with longer scales but we find that they add complexity more than they provide information.

Should there be a middle point?

Yes.
We know the fear - everyone will score in the middle (the Veblen effect). But they don't - really, they don't. And often people have a "she's fine", "ambivalent", "mixed feelings" type
response that they want to give. Forcing them to go positive or negative creates a false response.

Is “not applicable” allowed?

Yes.
You might call it something else : "no evidence"? But in 30 or so questions we have to expect some people to just not know.  Now - most importantly, you should review the responses to questions to ensure that questions are not overloaded with N/A responses - or that one group of respondents are answering in this way. If they are then the question requires rewording or removing.

The words : We favour scales such as "Strength", "Development Area" rather than "Often","Sometimes".

The latter scale, citing frequency, are however effective and really this is just personal preference. I find that the behaviours read better as a statement rather than as a question and this fits with a statement based rating scale. What I would avoid at all costs is
a numerical rating scale.

Overall thoughts

The rating scale in 360 degree feedback is less important than people make it - I believe that respondents look at it initially to get a feel for where to place their mark but really they are providing graduated feedback more than specific.

When debriefing though you do tend to use the rating scale as part of the language - and for that reason we concentrate on how the report will look when we are designing the rating scale rather than how it appears in the questionnaire.

Want to read more?  Download our 360 degree appraisal white paper.


Brendan

A 360 degree feedback questionnaire falls out of your competency framework.  Here are our rules of thumb for your questionnaire.

• Between 20 and 40 questions is about right
• Usually we see these as groups of 3-5 questions per competency
• Adding a narrative question for each competency is normally the right way to go.
• Consider whether every group of people can answer every question. If not, then exclude the questions from that group. So, peers may answer a subset of the overall question set for example (and so reduce the burden upon them).
• It can be useful to have a question (or even 2 or 3) at the end of the 360 that asks people to give broader feedback or cover points they would like to make
• Questions should be brief, clear and unambiguous, and describe an observable behaviour
• If you have people for whom English is not their first language then we would recommend translation of the questions. You probably don't need to translate the whole system but the nuance of questions matters

Questionnaire design can be a technical area - but if you stick to relatively short, clear questions then you can avoid issues that can flow from having complex overlapping questions which ask about more than one behavior.  As ever, simplicity is crucial in 360 degree feedback.

Brendan

In previous posts I've discussed the principles of a 360 degree feedback focused competency framework and I've given some rules of thumb.  Here is how I would go about creating a framework in practice.

• Start with your values and stated strategy - look to existing materials and language (collect any relevant documents/posters etc)
• Get directors, or senior management to express their desired behaviours – a facilitated workshop around suggested behaviours is an excellent way of drawing this out
• Seek to identify best practice in each area
• Involve managers and staff, outside of HR, in design and implementation - a series of workshops works brilliantly in reviewing your first drafts
• Keep it simple : use straight-forward language
• Communicate, communicate, communicate
• Train people in how to use, understand, and assess against the competencies which often challenges the core of the organisation.

By involving people through the process you will ensure that when you create a question set for your 360 feedback the respondents and recipients will recognise the language and feel that the questions are highly relevant.

This is an excerpt from our 360 degree appraisal white paper.


Brendan

These rules of thumb are for a 360 degree feedback focused competency framework.

• Less than 12 competencies per role
• 3 or 4 behaviours per competency works well
• Cluster the framework for ease-of use
• Contain both definitions and examples to aid understanding
• Tailor off-the-shelf frameworks where you can. Re-inventing the wheel is highly unnecessary for a lot or roles
• Create a forward-looking framework. What behaviours does the organisation want/need rather than what do they currently have
• Seek out best practice in each area

In my next post I will discuss how to take on the task of creating the competency framework for your 360 degree appraisal project.

Brendan

Continuing with our series of posts on implementing a 360 degree appraisal project, let me turn to competency framework design.

Competency frameworks are used by organisations for a variety of tasks – to support recruitment, promotion, career paths, development, training and more; they are clearly a topic in themselves.

For our purposes we are looking for the principles and a checklist for a competency framework that will support a 360 degree appraisal.

A competency framework should cascade from company objectives and values.

Without them you will generate a framework that is full of good intention but would equally apply to every other company and is unlikely to lead to behaviour that improves your organisation.

For 360 degree appraisal, the competency framework should be the simplest, cleanest framework that delivers the strategic intent of your 360.

Completeness is less important than focus. If the framework is only being used for 360 purposes then you only need the competencies and behaviours that you are currently  focusing on.

We firmly believe that the process of generating the framework is valuable in itself – asking managers and directors to consider which behaviours will lead to success is a fascinating exercise

In my next post I'll add some rules of thumb for the competency framework design.  Please feel free to subscribe to the blog (link in the top right hand corner) or if you want to get the full 360 degree feedback paper then please click here to download it.

Brendan

This checklist should be used to ensure that your 360 degree appraisal report is designed with the correct influences.

• What is the purpose of the feedback? In particular we are heavily influenced if the 360 is linked to an annual performance appraisal.
• Who will receive the report : the recipient, their manager, HR ? Does the information provide these stakeholders with the information they require?
• What experience has there been of 360s within the organisation? A simple 360 report is nearly always best and it is crucially important in the early stages of introducing 360.
• What is the culture of the organisation? Numbers and statistics can be grabbed and manipulated in hierarchical organisations.
• What icons, symbols, other literature exists within the organisation that should influence the design of the report.

With these influences in mind, the following structure usually works well.

• Summary table of who has given feedback (e.g. 3 peers, 4 direct reports)
• Overall, by competency, tabular summary of feedback (not averages) so you can see the big picture
• For each competency, a tabular summary of feedback
• Within each competency, graphical representation of feedback for each behaviour with a summary of the narrative comments received

You will spend the majority of time in a debrief session on the detail.  If you are interested in seeing some examples of how we design reports that do not use averages then please contact us.

Brendan

Continuing this series of posts on how to get a 360 degree feedback project up and running, let's look at the design phase.

The design phase covers reports, competencies, and questionnaire design. We try to start with the report – by doing so, you recognise that it is the output of the process that is important. 

The competencies are very important, but if you start with how the report will be structured, it focuses the mind on how these competencies will be used.

One of our key principles is that 360 degree appraisal is a method of consolidating feedback rather than averaging it; consequently, this influences everything we do so let me spend a little time on it.

360 is useful when it describes to you the range of responses that your colleagues give against observable behaviours. That some people think you are a star at delegating while two of your direct reports find your approach troublesome, is the value of 360.
So, only create scores or averages where they give value rather than obscuring information.

Second, we believe that narrative feedback offers the richest information - the rating scales simply tell you where to look. Anyone who has given 360 feedback will tell you that the comments that raters provide give you the evidence and narrative that explains or gives context to the behavioural scoring.

With these principles in mind, in my next post I'll draw out the detail of how we go about designing the report (competencies and questions will follow later).

(This series of posts are excerpts from our 360 degree appraisal white paper).

Brendan

Here is a checklist for a 360 degree appraisal project.  You're bound to need to add to it - but its a great starting place.

• Gain strategic direction on purpose and intent of the 360 feeedback process from senior management and write a short synopsis of this that is agreed by senior management
• Source any suppliers if required
• Run a short workshop with all stakeholders who can make key decisions for the project (e.g. anonymity, who picks raters, confidentiality)
• Identify any technical requirements and alert the I.T. Department to these requirements as they may have lead times (not applicable for our system but may be for others)
• Develop a communication programme and plan in
• Determine training requirements and plan in
• Design competency framework with the strategic intent and eventual reporting in mind
• Build the questionnaire and reporting that work with the competency framework and strategic purpose
• Plan any administrative resources that are required and begin to fix the implementation timetable
• Train raters and recipients directly or through communication plan
• Start process and send invitations to participants
• Monitor progress, remind and chase
• Train 'De-briefers' in how to share the 360 degree feedback report with the recipient
• Debrief recipients in a face-to-face session
• Run a review of the whole process attended by all parties who have been involved

This checklist is from our 360 degree appraisal white paper.


Brendan

This is an excerpt from our 360 feedback white paper.


A 360 degree appraisal project is often a concerted effort over a reasonable period of time. For organisations introducing 360 degree appraisals for the first time it requires an investment in communication, training, and co-ordination as well as the administration of the 360 process itself.

The diagram provides a simple picture of the key areas that should be considered.



360 degree feedback is not complicated but if you want the recipients of the feedback to have the best possible experience and to gain the best possible feedback then a careful,  well thought out approach will give you the best results.

A 360 feedback process should not be an isolated piece of work – it should be recognisably part of an overall plan for the individual and for the organisation. It can be a one-off exercise to support one manager's development or it can be a widespread initiative to improve an organisation.

In either case it should be in context and a little planning and consideration will go a long way.

360 degree appraisal initiatives are successful when the key stakeholders understand, believe in, and contribute to the 360 plan.

A 360 should be measuring behaviours that are valued by the organisation, the participants should understand how the 360 fits into their overall development plan, and the 360 report should lead to activity that is planned into the learning and  development plan.

In my next post I'll include a checklist for what you may want to include in your plan.

Brendan
 

If you are starting a 360 degree appraisal project this year, then I'll be writing a series of blog posts over the next few weeks that give some of our guidance.  Feel free to subscribe to the blog or, download our 360 feedback whitepaper now if you want to get everything in one go or can't wait!

Brendan

One of the questions on 360 degree feedback rating scales is - should we allow a 'not applicable' option?  It may be phrased as "don't know" or "no evidence" - the question is should we let people not give a full response.

We have always argued that you should and to the fear of some clients' that it will be over used, we have responded - don't worry it just isn't.

As we have a databank of millions of responses to 360 degree feedback, today I checked that out.  Over all of our millions of responses we have a 6% selection of 'n/a' or a similar option.  About what I expected.   And I can now say "don't worry about it" with more confidence.

One thing to point out - don't be too casual about this.  As part of your review of a successful 360 degree feedback process we would always recommend that you checked out how many n/a's you received on each question and from each respondent group.  You may need to remove or reword a question if people are unable to give a response.


Brendan
 

If you are looking to implement a 360 degree feedback process in 2010 then feel free to get our white paper on that subject.  360 feedback lends itself to guidance rather than rules - what works for you may not work for others.  Our white paper is written as a series of checklists to allow you to be certain that you have considered all of the important points but not be constrained by a set of zealous rules!

Download a copy of How to successfully implementa a 360 Degree Feedback process in your organisation.

Brendan
 

One of our best projects of 2009 was a full 360 feedback process for a law firm.  We worked with a consultant to the firm and helped end-to-end from design of the questionnaire through an on-line solution to debriefs and finally to present overall findings to the department heads.

During the project, the two sponsors within the firm had to take key decisions on how to implement the 360 feedback.  I saw a key decision as "do we need someone (external) to debrief the report with each recipient?".  This was certainly a big budgetary decision but it also required thought around what was the purpose of the 360.

The sponsors went for external debriefing which we then provided together with the consultant.  I know they made the correct decision.  The people involved in the 360 were all senior, many were unused to the process, many had never had feedback on their behaviours in this way before, and they all had great potential.  The feedback sessions were nearly all challenging as the partners worked hard to understand the feedback and the implications.  The client extracted amazing value out of the 360 by following it through rather than leaving the report, debriefing, and feedback to chance.

When reviewing the budget for a 360 feedback intervention I would encourage anyone to be very careful around how they ensure the report is delivered and reviewed.  Money spent up until that point is all spent on preparing for that moment.  Getting the report delivery and feedback right is crucial.  For 360 feedback for a senior team I would wholeheartedly recommend external debriefers.  For larger 360 feedback interventions then ensure that training programmes are in place for the people debriefing the report.

Brendan
 

If you are considering changing your performance appraisal process in 2010 can we humbly suggest you take a read of our "How to implement performance appraisals" white paper.  It is written to provide high level assistance and get you off the ground.

Download the How to successfully implement a performance appraisal in your organisation white paper.

Brendan

I was recently working with a client who is considering how to gain feedback on the leadership team against a new set of leadership behaviours.  As we discussed the best routes open to us, unsurprisingly, 360 degree feedback was looked at.  A bunch of assumptions then flowed through - not just for the people at the meeting, but for the people who would be participants in the process.

Your perception of 360 degree feedback is, for the majority of people, going to be driven by experience.  Those of us who work in the field will have experienced many different ways of getting feedback from a range of sources in many ways but for most people 360 feedback will be driven by how it was recently done "to" them.

The most common perception is of measurement and that measurement leading to a score - with charts, graphs, means, etc.  That score is often then benchmarked in some way.  But that doesn't have to be the way - indeed scoring in 360 at an individual level is difficult at best.  

I started to reflect on what 360 feedback really is.  At its essence, it is a route to collecting feedback from a range of sources (I guess to strictly be 360, then it has to be from direct reports, managers, self, and a peer group).  There is no requirement for that to be ratings based - it just typically is.  

It is quite possible that a group of people do not need or can not make use of a rating based 360.  But seeking narrative feedback from a range of people may be exactly what would help them to reflect on their impact on others. 

Before we make the leap from "it would be good to get observations from a range of people" to "and so we will get an average score from a range of people" we should at least pause and challenge that assumption.  It doesn't have to be so.

Brendan
 

Most performance appraisal processes encourage participants to create SMART objectives (specific, measurable, accurate, realistic, timely).  The idea came straight out of a belief in management by objectives - the belief that if you set stretch targets/goals for the year and monitor progress and reward success then the individual and organisation will move forward and achieve the strategic goals.

When you work on performance appraisals in a number of organisations, you find that not everyone can easily create SMART objectives.  You often then find people forcing their day job into an objective structure to comply with the process.  Nothing then follows from this process until the end of the year when the objective is reviewed.  Even if there are regular 1-1 meetings throughout the year they will not refer to the objectives as they were not truly driving activity.

I believe that SMART objectives only work in situations that suit mid to long term targets or project based activity.  For the many people for whom their role is repetitive and the requirement is to deliver consistent levels of quality or service creating an objective can be a trite activity.  Performance appraisals remain valid for this group of people but more on a competency and activity based review than objective based.

If you are not going to manage someone against their objectives - then don't set any.

Brendan

We're now fully prepared for our seminar on "how to successfully implement 360 degree feedback".  The seminar is running this Friday (11th December) morning in central London.  Both John and I will be presenting and we have spaces remaining if you are interested.

To attend, please register on our website and we will get details of the venue over to you.

The session will be a mix of theory, principles and some practice.  If you are involved in 360 feedback implementations in 2010 then come along.

Brendan

I wrote a few weeks ago about a comment that had been written on my wife's blog.  The comment was highly encouraging and supportive.  Continuing an odd theme of my family's internet activities, I have an uncle with a photography website.  In talking with him at the weekend I was struck again how reliant we are on positive feedback.  The best feedback he gets is when someone purchases a photograph of his but more generally some comment or even taking the time to comment makes a massive difference.  We have a picture of his on the office wall and a recent visitor commented upon it and I passed this compliment on.

Positive feedback can sometimes be harder to give - or harder to remember to give.  Most of us focus on what can be improved and are never quite satisfied.  Note to self though - positive feedback whether as part of 360 degree feedback or as part of general performance reviews is highly powerful.

Brendan

I found myself trying to answer the question "should we use a rating scale of 1 to 10 or 1 to 5 for our 360 degree feedback.  Trying to be helpful, I started thinking through the merits of both approaches.  Easiest route to answer this type of question is to think about it from the respondents perspective.  And that is when you spot the problem.  I don't know what being a 3 out of 5 or 10 at "delegating" or "client service" means.  It just doesn't work.

We often encourage clients to not use numbers in their reports - our belief is that averaging at an individual level usually loses more than it gains.  Using numbers for the respondent is almost impossible to implement.  So, the answer is use a worded rating scale.  5 points normally works with a not applicable/no evidence option.

Brendan