Latest Articles » How to tell you’re asking the wrong question

How to tell you’re asking the wrong question

As you read this, you will be preparing to vote in one of the most skewed referenda this country has ever seen.

Perhaps more likely, you have decided not to vote because the question itself makes the whole exercise a farce (quite apart from the fact it is a non-binding referendum).

Regardless of your views about the so-called anti-smacking legislation, the referendum being conducted from 31 July to 7 August is a very public example of how leading questions make responses worthless:

"Should a smack as part of good parental correction be a criminal offence in New Zealand?"

We might as well spend $9 million asking:

"Should lobby group petition organisers be allowed to define the wording of referenda questions with the intention of yielding a particular answer?"

Before letter-writers on either side of the debate begin generating a head of steam, my point is this - if the purpose of a referendum is to accurately gauge views on an issue and preferences for a course of action (e.g. changing a law or leaving it as-is), then questions such as this fail to achieve those objectives.

It is not an issue of clarity. The question is clear. It is simply that it is leading (i.e. designed to yield a particular answer).

For  an example of an unclear question, see the NZ Herald online poll which asked:

"Do you support the referendum on child discipline laws?"

What does that mean? If I was to "support the referendum", how exactly would I do so?

Is it in fact a referendum on child discipline laws? Some might disagree. If it is, what options am I being asked to support or oppose?

The UK's Mail Online news website provided a good (or should that be bad?) example of a leading question, with a poll asking:

"Should the NHS (the UK's National Health Service) allow gypsies to jump the queue?"

The question attracted ridicule amongst Twitter users who banded together to swamp the site's servers with unexpected 'Yes' votes in a bid to draw attention to the question's lack of neutrality.

Faced with 96 per cent support for the idea (contrasting with an editorial line opposing anything it considers liberal) the Mail Online quickly removed the poll.

Which leads us (pun intended) to the relevance to business.

Business needs a constant flow of reliable intelligence - about customer perceptions and preferences, price and value comparisons, and purchasing intentions.

In a bid to get to the heart of an issue, businesses (and the occasional market researcher) sometimes pose questions which fail to yield reliable information. How can you avoid falling into the same trap?

How can you tell you're asking the wrong question, or asking the right question in a way that's biased towards yielding a particular response?

The first indicator of bias is an obvious (no-brainer) answer, coupled with a highly-unlikely opposite response.

The second is wording which does not clearly define an issue, or offer a full range of response options. Even "don't know" and "don't care" are usually reasonable options.

A wrong question is a little harder to spot. But a sure giveaway is that a leap of assumption is required between the research and its practical application.

At first glance, an assumption may appear reasonable or even accurate, but needs to be challenged.

Why was an assumption necessary? Could reliable information be obtained from a direct question?

This is one area where there is no substitute for experience and professional detachment. Unless of course, you want a particular answer . . .

We might as well spend $9 million asking: "Should lobby group petition organisers be allowed to define the wording of referenda questions with the intention of yielding a particular answer?"