I was working on the wording of a 360 degree feedback questionnaire and in an idle aside I came to think of how the way we word the question and the order of them can lead to a different response. Which got me hunting the internet for a Yes, Prime Minister (UK sitcom) episode. Here is the dialogue.
Sir Humphrey Appleby: Mr. Woolley, are you worried about the rise in crime among teenagers?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think there is lack of discipline and vigorous training in our Comprehensive Schools?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think young people welcome some structure and leadership in their lives?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do they respond to a challenge?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Might you be in favour of reintroducing National Service?
Bernard Woolley: Er, I might be.
Sir Humphrey Appleby: Yes or no?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Of course, after all you've said you can't say no to that. On the other hand, the surveys can reach opposite conclusions.
[survey two]
Sir Humphrey Appleby: Mr. Woolley, are you worried about the danger of war?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Are you unhappy about the growth of armaments?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think there's a danger in giving young people guns and teaching them how to kill?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think it's wrong to force people to take arms against their will?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Would you oppose the reintroduction of conscription?
Bernard Woolley: Yes.
[does a double-take]
Sir Humphrey Appleby: There you are, Bernard. The perfectly balanced sample.
Now, this is an obvious extreme but sometimes, particularly on employee surveys but occasionally on 360 degree feedback questionnaires I see the line of thinking of the survey writer shown up in the order and shape of the questions.
Brendan
Sir Humphrey Appleby: Mr. Woolley, are you worried about the rise in crime among teenagers?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think there is lack of discipline and vigorous training in our Comprehensive Schools?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think young people welcome some structure and leadership in their lives?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do they respond to a challenge?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Might you be in favour of reintroducing National Service?
Bernard Woolley: Er, I might be.
Sir Humphrey Appleby: Yes or no?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Of course, after all you've said you can't say no to that. On the other hand, the surveys can reach opposite conclusions.
[survey two]
Sir Humphrey Appleby: Mr. Woolley, are you worried about the danger of war?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Are you unhappy about the growth of armaments?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think there's a danger in giving young people guns and teaching them how to kill?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Do you think it's wrong to force people to take arms against their will?
Bernard Woolley: Yes.
Sir Humphrey Appleby: Would you oppose the reintroduction of conscription?
Bernard Woolley: Yes.
[does a double-take]
Sir Humphrey Appleby: There you are, Bernard. The perfectly balanced sample.
Now, this is an obvious extreme but sometimes, particularly on employee surveys but occasionally on 360 degree feedback questionnaires I see the line of thinking of the survey writer shown up in the order and shape of the questions.
Brendan





Comments for Yes, Prime Minister - the art of surveys