How this links to 360 degree feedback will follow!

Let's assume we have a basic traffic light system.  And we find out that we have the following distribution

• Red 50% of the time
• Amber 10% of the time
• Green 40% of the time

And someone wants to know what colour it is on average.  What to do?

Average requires numbers.  

So, let's give Red the number 1, Amber the number 2, and Green the number 3.  A bit of Maths will find the average now (50% *1) + (10% * 2) + (40% *3) ... 1.9 is the answer.

So our average is 1.9 which is nearest to Amber (which we gave the number 2).  So on average the colour of the traffic light is Amber ... somewhere in the middle.

We know this is wrong - the light is on amber least of all - but it was an attractive solution somehow. 

Even more tempting is to ask people to respond to a question with responses that can be

• Strongly Agree
• Agree
• Neither agree nor disagree
• Disagree
• Strongly Disagree

and then give people a score out of 5 for how people answered on average.

Let's say we have done that and the average is 3.1.  What does that mean?  We're going to say that on average the respondents roughly "neither agree nor disagree" with the statement.  But go back to our traffic light example... the colour that the traffic light is least often is amber, what's to say the same hasn't happened here?

Statisticians will tell you that the underlying problem here is that you are treating categorical data as if it is numerical data; if you are very unlucky, you may find yourself in a debate about likert scales and the Polytomouse Rasch model.  What I want to highlight is, this superficially simple concept of giving each point a number and averaging the responses is not that simple and may lead to inaccurate conclusions.

Way's round it?  Either don't use averages in your report, or make it clear that you are using the scale to reflect a mark out of 5.  That at least gives your average credibility even if it doesn't get around the problem of the average obscuring the underlying scoring. 

Note, this doesn't stop you using numbers or charts in your report - I'll discuss that in another post.

This article forms part of the structure of a new white paper I am writing on reporting in 360 degree appraisals (I promise this is the heavily statistical bit and the rest is more down to what we see as best practice!).  If you sign up to our white papers, then you will receive that document as it is completed.  Click here to sign up.



Brendan

Hal Varian, Google's chief economist, is quoted as saying that "Data are widely available, what is scarce is the ability to extract wisdom from them".  I'm focusing heavily at the moment on debriefing 360 degree feedback and the 360 degree feedback report.  For both an upcoming seminar, and a new whitepaper, I'm looking to fine tune our thoughts around how we make best use of a 360 feedback process.

The current line of thinking is to consider how data becomes information becomes knowledge/wisdom.  The 360 feedback questionnaire generates data.  Our challenge is to take that data and produce information from which the recipient gains knowledge.  Along the way we have to avoid the dangers of losing information or of forming unwise conclusions.

The report and the conversation around the report is where the transformation happens and where best practice can lead to the best knowledge outcomes.

You can register for our 360 feedback seminar by clicking here.  If you are interested in our white papers then subscribe here - you would then automatically receive the white paper described above as it is produced.


Brendan

360 degree feedback deliberately generates data from a range of sources - it creates more data than a standard performance appraisal.  Annual performance appraisals are also starting to commonly seek information from a range of sources - it reflects a move to more networked organisations and less structured boss->subordinate relationships.

That all makes sense to me - it is a sensible growth in data.  But when it comes to the annual performance appraisal working out what is worth measuring is important.  I've been thinking about this a lot as I've been looking recently as my wife develops a new business promoting deals and discounts for days out in the UK.

Her website has Google analytics that tells her how many people visit the site, which pages are popular, etc.  The blog http://blog.topdogdays.com tells her how many people have subscribed to the blog.  Her twitter service http://www.twitter.com/topdogdays tells her how many people are following her.  Amazon tell her how many people have bought a book having visited the site, and google tell her how many people have clicked on an advert on the site.  Data, data, everywhere.  Eventually all of this data can distract from the purpose of the business and managing it.  But it is highly seductive and of course in the early days it is great feedback.

For all of us when reviewing performance - or setting the targets for next year - it is critical to boil down the measures to the key performance indicators.  A term that makes a lot of sense but is often abused.  We need to watch the key performance indicators - not all of them.  In a previous life, I ran a call centre operation.  We had stats coming at us from all directions - all that really mattered was 1) did we answer the calls and 2) did we provide a great service when we did.  Ring time, call duration, "not ready" time, and hundreds of other numbers were indicators but not key indicators.

Brendan

I was listening to Radio 5 in the UK.  There was a debate about class - a standard fixation.  Initially a couple of experts in the field discussed the issues around class and social mobility, with the presenter facilitating the debate.  All good so far and I was learning stuff.  Then "Chris from Manchester" called in.  It is a feature of many news avenues now that we are all invited to contribute.  Chris wasn't an expert, he was a perfectly fine person with an opinion and I was now listening to it.  It wouldn't be long before another 20 people would call in with their opinion and I began to learn less and less.

In 360 degree feedback we seek feedback - opinion - from a variety of people.  We sometimes get a desire to garner feedback from many, many people.  The concern is that if we don't ask everyone possible then 1) someone will feel left out and 2) we will miss a piece of valuable feedback.  At times, for certain clients, these are valid concerns and the solution is to get feedback from 20-30 people.  But usually, getting feedback from too many people has two problems.

First, we get the "Chris from Manchester" problem.  Chris feels the need to offer an opinion even though he's not an expert (read; he doesn't actually know this person that well).  Chris writes a lot of commentary and his scoring gets equal weight with people who spend every day with the 360 feedback recipient.

Second, when we get the 360 feedback report back, we can't see the wood for the trees.  It's like spending the day on Twitter.  You know something important must be in here but you can't find it because of the volume of data that is coming to you.

So - unless you have specific reasons (normally cultural concerns) that you have to be aware of - be selective in who gives feedback.  The quality of your knowledge will improve.


Brendan

One author wrote that if you want to make performance appraisals really difficult then link the individual's pay to their numerical rating.

Without judgement, we take the position that some organisations wish to use the performance appraisal process to help them determine the level of renumeration – salary or bonus – of individuals. If that is the  case, then how should the performance appraisal process be run to best achieve this?

First, let us consider what is a good outcome. We would argue that a good outcome for the advocates of performance related pay is

• Individuals motivated to achieve targets that will improve the organisation and meet the organisational strategy.
• The correct people getting the correct rewards
• An efficient performance review process that delivers the benefits without using those benefits up in administrative burden
• A robust process that stands scrutiny from external parties particularly on equality

When you consider the list above you are immediately struck by the need to get the start right. It is not so much the system of calculating rewards that matters – more it is a matter of ensuring that the measures are generated well. Better that our grading structure is simplistic than we skip past the step of generating fair targets.

So, first and foremost if you are looking to implement performance related pay and are using performance appraisals to support that implementation – spend a lot of time thinking about how to get the measure right. Continuing our humble theme of not knowing what is right for you, let us describe some options that we have seen work.

• Weighted objectives, agreed between manager and employee and cascaded from the organisational strategy and graded for achievement.
• Value statements derived from the company values and graded for compliance.
• KPI indicators developed in consultation with employees
• Monthly targets, adjusted each month against which employees are graded/scored each month
• Team/Organisation objectives against which whole team's are measured
• Survey based data – e.g. customer satisfaction scores, against which individuals and teams are reviewed

Before embarking on performance related pay we would advocate a thorough consideration of what you are looking to achieve.  If you decide that it is right for your organisation then I hope you find this note useful as a first step to delivering a robust process.


Brendan

The appraisal meeting strikes fear into many managers. They fear its time-consuming nature and they fear the meeting itself. The former issue is often cultural. The time spent on performance appraisals is a fantastic investment for managers if the process is run well. A  stitch in time saves nine.

The meeting itself is only feared by poorly trained managers who are uncertain of how to handle the meeting. Appraising an individual is an unnatural task for many managers but it can be trained.

The structure of our own training course for the performance appraisal meeting is as follows.

• Understanding the purpose of performance management and the annual cycle
• How to review performance in-year
• How to conduct the end-year performance appraisal meeting
• How to handle performance and behaviour problems
• Use of coaching within appraisals: the GROW model
• Core skills: listening, asking questions, giving feedback, confronting, supporting.

Contact us if you are interested in this training course, or if you would like the performance appraisal white paper that this blog post is an excerpt from.


Brendan

I was reading a book on long distance cycle training.  The author referred to SMARTER objectives.  The acronym was used differently, although the SMART was very similar to the HR useage.

Specific
Measurable
Agreed (he referred to sharing your goal with someone else)
Realistic
Time-phased
Exciting
Recorded

The two latter lines are the new ones to me.  Recorded is fine - it just makes sense, but exciting catches the eye.  An exciting objective in an performance appraisal sound far-fetched?  But why not?  And if exciting is too far - surely interesting is something we could look for?

Too often you see dry goals that are unlikely to drive someone to higher performance or gain personal reward from achieving the goal.

I've started training to complete a cycle ride from Land's End to John O'Groats in the UK (about 900 miles) in 9 days.  A wonderfully SMART objective - although I have dark moments where realistic is in doubt!  And I find the idea so exciting that I'm motivated for training, buying books to learn about how I can improve endurance, and putting my own milestones in place to make sure I'm on track.

Worth a thought - a SMARTER goal in an annual performance review form could lead to considerable improvement.

Brendan

This is an excerpt from our performance appraisal white paper.

The common advice is that at the annual appraisal nothing should come as a surprise. Through regular feedback the manager should ensure that an employee is always aware of how things are going, where they stand, where their greatest achievements lie and where they need to develop. We agree with all of this. There remains a question of how often the organisation and the people involved want to formalise this process.

The benefits of the formalisation is it ensures noone is slipping under the radar, allows the organisation to get some data back that it can direct training and other interventions towards, and it supports company practices such as pay reviews, and promotions.

Annual is too infrequent – too much changes in a year. But every other structure after that is down to individual organisations. Our view is that commonly an interim 6 month review is what is needed for formal appraisal. A monthly meeting should be scheduled in as good practice but keep that unbureaucratic.

Brendan

It may be worth pointing out at this stage that these excerpts from our performance appraisal white paper is a suggested order of working rather than a suggested order of importance. Done properly the personal development plan should  be the most important part of the performance appraisal.   Assuming that one of the main intentions of this process is to have people improve and so lead to improved organisational performance then a development plan is the key.

The reason for the order is that we find that a practical consideration is whether the  development plan is part of the same annual appraisal meeting and form completion process as the  objectives, values, and scoring. Often it is not. Indeed, practical constraints aside we would suggest that the development plan is kept separate from the appraisal form itself. It  requires a slightly different mindset and lives in a different way.

In principle the development plan should describe the skills, knowledge, and behavioural  changes that the individual is looking to develop over the coming time period. It generally  follows that most of the development plan should flow out of the review of prior year  objectives / values and consideration of goals for the coming year. This is important and  needs guidance to those completing the form. That an individual does not know Spanish and  would like to learn the language is only relevant if 1) they need to know it for work or  2) the organisation has a value of broadening peoples abilities.

The performance appraisal form is commonly a general HR domain. The development plan must be produced in concert with the learning and development/training team. Much completing of  Excel spreadsheets can be eliminated by a well designed, online development plan.

This is an excerpt from our performance appraisal white paper.

Brendan

Your annual performance grade is something that sticks with you. It can overwhelm the whole process. But it is not the most important part of the process and indeed performance appraisal can work perfectly well without one.

If you are not looking to use the appraisal as a link to pay then consider long and hard whether you need this one line/number summary of the year.  We find it distracts the appraisal meeting and distracts the appraisal project itself – heavily influencing how objectives, values and 360s are designed and completed.

If you do need the grade (and most of our clients do) then let's consider our options.

Commonly we see two sorts of grade; a numeric grade (e.g. 1, 2, 3, 4) or a narrative grade (e.g. Strong performer, Competent, Development required). What we see less commonality  on is how this grade is determined.

End of the form grade

Still the most common, there is a simple drop down box of options that the manager selects  from.

Calculated average

Seemingly growing in popularity, we see grades calculated from other ratings on the annual  performance review form, or built from grades in interim reviews across the year. The most common is to grade how objectives have been completed.

Suggested calculated average with override

A late entrant, but increasingly a request, is to calculate an average within the system and  then give an option for the manager to override the calculation - normally with a forced  narrative option to explain the discrepancy.

Forced distribution

All of the above options can be subject to a forced distribution (e.g. 20% of people will be an  A, 40% will be a B, etc.). Some form of scoring drives this distribution which can be  across the whole organisation or across departments.

360 degree feedback influenced

From scoring on 360 feedback, the annual performance review grade or evaluation is  influenced or calculated.  Take great care with this option.

I have to say our view is not set in stone here. In an ideal world I suspect we would avoid  the annual grade - it can be distracting and it can be more controversial than it is useful. But, if you are going to run performance related pay (a debate in itself) then a grade is likely to form an element of the review process. I believe then that having followed a sensible  process, managerial discretion is required on the grading. Whether that is assisted -  through averaging - isn't actually that important.

With such an important topic area, training is the key to ensuring the managers are able to  apply the chosen process in a fair manner and in a way that achieves the performance  appraisal process objectives. Managers will need training and role playing that ensures that  the gradings being presented are even across the organisation. This training can of course  be included in training on how to handle the appraisal meeting.

The above is an excerpt from our performance appraisal white paper.


Brendan

Competency assessments allow the organisation to set a standard set of statements  against which employees are assessed. Generally those competencies are specific to particular roles.

We have clients who make extensive use of competency assessments – often with many  banks of statements for a wide range of roles. At the other extreme we have clients with just one competency set for all people. It very much depends on your organisation and on the time you have to dedicate to the task.

For an organisation of any size with anything but the most homogenous set of roles then  we would recommend competencies that are role or at least job group specific.

We find  competency assessments work particularly well in performance appraisals (as opposed to 360 degree feedback) where we are looking at roles with straight forward tasks. For example, a very  effective tick box structured competency assessment of a manual work role can be developed that is easy to complete and generates exactly the conversation and   development plan you are looking for. Often, this competency assessment is better than an objectives section for these roles.

Take some care if you wish to introduce a 360 feedback element to this section.  Generally 360 works best as a development tool rather than an appraisal.  Also, it adds a significant burden into the process.  Gaining feedback from a range of sources can be done in a simpler manner than a full 360.

As a point of detail : the rating scale for a competency assessment is suited to “strong to weak” rating rather  than frequency based. We suggest you use a set of words that are consistent with other  areas of the form or other wordings used within the organisation.

This is an excerpt from our performance appraisal white paper.


Brendan

A performance appraisal offers the opportunity to discuss the values of the organisation and how the individual in their day-today work is promoting those values. A values review serves two key purposes

• Reminds all involved in the process of the values and the importance the organisation places on them
• Highlights practices that are outside of the desired values

You may wish to consider who is best placed to comment on this.

Often the manager is not best placed. Colleagues and direct reports are more likely to see the actions of an individual particularly in remote teams. 360 degree feedback is often used to gain insight into behavioural indicators – although be careful not to lose 360's key benefit of being developmental. A good manager should have enough contact with the team and colleagues to have this insight without 360 – but it is an option.

A values review can be very subjective. Compared to determining whether a well-formed objective has been achieved or partially achieved, it can be more contentious to discuss  integrity or openness.

The aim of the process here is generally to prompt an open conversation between the manager and the individual so we need something simple. A clean, rating based assessment with overall comments can offer a quick route for the manager (and perhaps the individual themselves) to give an overview and then prompt a conversation. As a matter of detail, a frequency rating scale often works well here. It is easier to answer “Displays integrity” with
Often or Very Often than it is to say Good or Excellent.

The subjective nature of values reviews also lends a problem for using their scoring for an  overall score or link to performance related pay. The benefit of linking them to pay for a  number of organisations is that it demonstrates their importance. The organisation is saying  we don't only care whether you achieve the big goals we also care how you go about the work. You have to balance the inherently difficult nature of scoring values with the benefits of demonstrating their importance.  Our instinct is to not link it to pay but it's not a hard and fast rule.

This is an excerpt from our performance appraisal white paper.


Brendan

In my last post I asked that you at least question whether you need an objective section within your annual performance appraisal.  It is though the most common section, so let's look at it in some detail.

The basic tenet of objectives is that they should be SMART (Specific, Measurable, Achievable, Realistic, Timely). While, this is a useful place to start it is worth recognising that SMART tended to be used in project management or marketing circles first. We recommend that you set your objective questions with SMART in mind but you are not restricted to it – for example, the subtlety of achievable versus realistic is often unnecessary.

So, you want managers or individuals to set objectives that are specific and difficult (remember you goal theory). But, here is where the problems lie. It is not necessarily in the individual's interest to set difficult objectives. Not from an inherent laziness (I firmly believe that employees are not inherently lazy) but if the performance appraisal process has consequences then it requires a particularly brave  person who sets themselves a challenging target. We suggest that you follow the steps below to ensure objectives are well-set and that you generally have managers set  objectives rather than the individual themself.

This is not cast in stone, and we are not advocating this process happens without a  dialogue but in the end, the manager sets the objective.

Our general recommendation is that you reduce the time-period for goals. An annual goal is subject to a large number of external influences that make it unlikely to lead to sustained  effort. It is likely to be affected by influences outside of the appraisee's control, it is difficult to remain motivated to the goal should you start to over- or under- perform against it, and it  may need to change over the course of a year.

An attractive but elusive goal of objective setting is to allow the organisational objectives to cascade through the company. The theory is deceptively simple – we set the organisational strategy and the senior team set their objectives based upon it. The next level down set their objectives in order to achieve their manager's objectives – cascading down a set of  well aligned objectives.

Our experience of cascaded objectives in performance appraisals is less positive. First, you should consider the practical problems of the time taken to allow the cascading to take place. Most organisations struggle to let it happen. Second, it relies on objectives to be clearer and somewhat simpler than they tend to be. Third, it assumes omnipotence on behalf of the senior team – they need to get the strategy so right and their own objectives so accurate that others beneath them don't need to create objectives outside that are actually needed.

There is a simpler and we believe better way of achieving alignment without prescription in this way.  Our suggestion is that all objectives should be allocated against broad themes set by the organisation. Frankly, there are times when people set objectives that have little to  do with the business or focus of the organisation and the theming brings this in line.

In summary - keep this section simple: what am I going to do, when am I going to do it (not too far away), what do I need to help me do it and what measures am I putting in place.

Excerpt from our performance appraisal white paper.


Brendan

Continuing my extracts from our performance appraisal white paper, let me turn to objective setting.

Objective setting derives from goal setting theory. Goal setting theory states that people working to specific, difficult goals consistently outperform those without goals or those who are simply urged “to do their best”. This is important. If you believe in goal theory then you have to consider how best to bring this into the objective section of your appraisal process.

In my next post/extract I will give our views on how to set objectives but  I would encourage everyone to ensure that they do want objectives within their performance appraisal form.  Too often they appear by default and it is not clear that this theory is subscribed to.  It is perfectly possible that yours is an organisation that does not need personal objectives to get people to work harder or better.  At least think it through before getting everyone in the organisation to try to complete them within their annual performance appraisal.

Brendan

An oft-missed step. For many organisations, performance appraisals are a given – often written into the company's procedures manual. Because we rarely decide whether to do
performance appraisals we sometimes forgot to ask “why are we doing this?”.

First, we recommend that you explicitly separate the organisational objectives from the personal development objectives. While they overlap and of course it could be argued that they are the same, this split allows you to meet the two stakeholders needs openly.

Organisational objectives for performance appraisals

• Clarifying and defining performance expectations
• Facilitating communication and involvement
• Allocating financial rewards
• Determining promotion
• Motivating employees
• Controlling employees actions
• Succession planning
• Cultural change initiatives
• Training needs analysis

Individual objectives for performance appraisals

• Identify training needs
• Identify development requirements
• Gain feedback on performance
• Promote own capabilities to organisation
• Understand expectations

It may be that you do not agree with these lists and almost certainly you would have other objectives to add. Our point is that the building of this list is crucial for it is the yardstick against which you can evaluate your current processes, any changes you design and the final implementation.

Our suggested approach is that you hold workshops that discuss with the various stakeholders what their objectives are.  This often gives your project focus and it allows you to introduce performance appraisals later with the benefits that people are seeking.

This is an extract from our performance appraisal white paper.

Brendan

After a slow start to the year due to the weather, the pace at Bowland has picked right back up to where we left off in 2009.  The last 3 months have been very busy for us as we picked up a range of both performance appraisal and 360 degree feedback clients.  I'm not complaining of course and we're always interested in working with people who are looking at 360 or their annual review.

The point of this post was to acknowledge how difficult it is to work on the people side when tasks are rushing in at us.  Taking the time out to talk with team members, give feedback, listen to their requirements is counterintuitive when times are busy.  My recommendation is "put it in the diary".  Make it another part of the day, make it part of the important list of items you absolutely have to get to and don't, just don't, miss that appointment.  Over any period of time, the time invested in meeting with, talking to, and listening to the team brings its rewards.

Am I good at this?  Not great - but getting better.  

Brendan

Following on from the series of blog posts on how to implement a 360 degree feedback process, I now turn to implementing performance appraisals.

As before, this series of posts is taken from our performance appraisal white paper.  This can be downloaded by clicking here.  If however, you prefer to read/subscribe to blog posts then over the coming days, I will post various sections of the white paper with a final concluding post that brings it all together.

Performance appraisals are often very different to 360 feedback initiatives and we certainly approach them differently.

Brendan

I have just completed a series of posts on 360 degree feedback.  You can get those posts through our white paper which brings the content together in an orderly fashion.  Click here if you want that white paper.

Alternatively, I have collated the posts below so that you can read a particular one at your leisure - just bookmark this page and pick the one of interest.

Planning a 360 degree appraisal process
360 degree feedback checklist
Designing your 360 degree feedback solution
Checklist for a 360 degree appraisal report
Competency framework design for 360 degree feedback
Competency framework rules of thumb
How to write a competency framework
Checklist for creating a 360 degree appraisal questionnaire
360 degree feedback rating scale
Choosing an online 360 degree feedback system
Communication as part of a 360 degree appraisal project
The 360 degree feedback debrief session


Brendan

This is the final post in a series from our 360 degree appraisal white paper.  In many ways it is the most important in that it talks about the conversation at the end of the process.

We like to use the term 'debrief ' to refer to the session where a recipient receives their feedback. It is not a coaching session – its focus is simply to ensure that the report is understood and the recipient has the opportunity to ask immediate questions.

Our first principle is that the recipient should not receive the report prior to the debrief session.  The 360 report can be powerful and has the potential to be misunderstood. The ability to place feedback in context, avoid misunderstanding, and spot potential issues is a core skill of the debriefer.

Our second principle is that the debriefer is there to enable understanding not proffer opinion.

During the meeting there is often the opportunity to challenge the recipients interpretation of the feedback but that should come from a standpoint of seeking understanding not of offering solutions

Here is our structure for a debrief session.

• Introduction and welcome
• Explain the purpose of the 360 feedback debrief
• Get the recipient to briefly describe their role
• Briefly describe how the report is structured
• Hand over the report and invite the recipient to skim-read it
• Ask for their overall reaction
• Review strengths
• Review potential development areas
• Identify actions
• Agree next steps
• Ask for feedback on the 360 process and the debrief

Brendan

Communicating effectively will reduce anxiety and increase participation.

We believe everyone needs to understand the context of a 360 degree feedback process and so well-written, timely communication at the beginning of the process will make the rest
of the process smooth.

However, when actually in the 360, we believe that you should communicate only when absolutely necessary. Automated emails going out every day will only serve to de-sensitise the recipients to your communication

Your initial communication should cover:

• Why are we doing a 360?
• What are the outcomes / what will it be used for?
• What is the process?

At each phase of communication you can touch these points rather than rely on one lengthy email or information document that may not be read.

How and when to communicate is really dependent on the current knowledge within the company, the abilities of the participants, the culture, and the level of complexity in the process. As a minimum we would recommend:

• Warm up communications that identify that the 360 is coming, the intention of it and at a high level how it will work
• With a week to go, we recommend an email that explains the process
• Within the email invitations to the recipients and/or the respondents of the 360 feedback, again give process information and clarity of purpose

Brendan