Chapter 4 Neutral scores matter

You might have noticed in some surveys that there is often no “neutral” or “undecided” category included in the middle of the scale, e.g., what’s usually a 3 on a 5-point Likert scale. Sometimes it is placed at the end of the scale, and sometimes it is eliminated entirely. The reason for this is that those terms can sometimes be interpreted in a variety of ways; for example, with a question such as “My pay is fair compared with other companies”, a Neutral response could indicate “I’m neutral on this”, “yes, I guess so”, “I don’t know”, “it’s neither fair nor unfair”, “I don’t want to answer”, “I’m not sure what ‘fair’ means”, and any number of ideas that don’t necessarily indicate a true neutral opinion.

When a question has a response option where this type of ambiguity exists, a mean value will tend toward the that option because of this bias, unless of course the mean is already at that value. However, when Neutral is marked as 3, and when valid responses tend towards 4s and 5s, these ambiguous responses will drag down the average (and vice versa for responses heavy with 1s and 2s). Of course, you shouldn’t use means anyway, as we’ve seen above, but many reports do—so understanding this effect is important toward interpreting the results in a useful way.

Use of a median is somewhat resistant to this problem, though you still won’t know whether the middle values are valid responses or accidents of interpretation.

When you see an “undecided” or “N/A” response placed at the end of the scale or missing entirely, it is usually (but not always!) a sign that the survey creator understands this problem.

Sometimes, of course, Neutral can be a completely reasonable and unambiguous response to a question. Context matters; while it’s easiest for survey creators and scanning software to use the same scale for large numbers of questions, it is important that the analyst understand the extent to which Neutral and similar types of responses are a valid part of the measurement scale for each question.