When doing consumer research, the questions you ask will have a significant effect on the results you receive. That is why Cambri will never let you ask a question on a scale of one to ten.
Why not? Because concept testing is only useful if you can trust your results. And according to evidence from marketing research, some Likert scales are just more accurate than others.
Market research shows some Likert scales are just more accurate than others
So, if the 10-point scale is a no-go, how do you design a concept test that uses Likert scales properly?
The short answer: use a 5-point scale. Studies show that five response options are best for questions that address the general public.
Read on, and we’ll explain why.
Today, Likert scales are nearly synonymous with “rating scales.” You’ve seen them a thousand times. Basically, they’re the go-to method for measuring a person’s attitude to a given question. That's why Likert scales are very popular when looking for easy-to-interpret consumer insights.
But not all scales are born equal. A 10-point scale is highly unreliable in terms of the validity of its results.
For instance:
How likely is it that you would buy this product on 1-10, where 1 = I would definitely not buy it, and 10 = I would definitely buy it?
There are just too many options.
Instead, here are three Likert scale best practices to follow when you are about to start concept testing.
1. Don’t force people to choose sides
Consumer research shows that sometimes people just don’t have an opinion. They’re neutral or indifferent to the options you’ve presented.
For instance, if you’re trying to decide between packaging design options, knowing that your customer doesn't notice or care about the difference between the expensive version and the cheap one is very valuable information. So do not regard indifference as a non-response. The best consumer insights come sometimes from knowing what consumers care about and what is irrelevant to them.
Takeaway: Never design your Likert scale with an even number of responses (i.e., 4-point or 6-point), because you’re asking people to select an opinion they might not really have.
2. Make space for a diversity of opinions
Although there's no reason to give people too many options, giving too few might also prove to be problematic. According to a 2010 study by Weijters, Cabooters, & Schillewaert in the International Journal of Research in Marketing, a scale needs to provide enough options for people to give their true opinions.
For instance, a person might be very convinced to buy something but still need a bit more reassurance before they actually make the purchase. If the options you give them are only 1 = would definitely not buy it, 2 = I am not sure if I would buy it or not , 3 = would definitely buy it, the person might not be able to express themselves correctly.
Takeaway: A 3-point Likert scale is too narrow to capture subtleties of opinion. That's why you should give more options than three.
3. Don’t confuse people with too many similar options
On the other hand, the larger the response scale, the more cognitive effort people have to use to pick an answer. Response scales that go up to 7 or even 9 options are challenging to interpret.
This can encourage people to give arbitrary answers, rather than honest ones. If a person is inclined towards buying something but not yet completely convinced, they might place their answer somewhere between 5 and 10 on a 10-point Likert scale. It might be very hard to interpret the nuances between points such as 7 and 9 and the results won't therefore provide valuable information.
Takeway: Research shows that a 5-point Likert scale gets the most valid results. It’s the Goldilocks of Likert scales. That is why it is Cambri's preferred method and recommended for anyone looking for high-quality consumer insights.
Sources:
Nowlis, S.M., Kahn, B.E. & Dhar, R. (2002). Coping with Ambivalence: the Effect of Removing a Neutral Option on Consumer Attitude and Preference Judgments. Journal of Consumer Research, 29, 319-334
Swain, S. D., Weathers, D., & Niedrich, R. W. (2008). Assessing three sources of misresponse to reversed Likert items. Journal of Marketing Research, 45 (Feb), 116-131.
Weijters, B., Cabooter, E., & Schillewaert, N. (2010). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27(3), 236-247.
Cambri’s concept testing tool helps you ask the right questions using science-based guardrails, so that you can be confident in your results. Get a 2 minute demo now!