Randomizing Options for Multiple Choice Questions

As a survey creator, one of your key jobs is to reduce bias that might creep into the responses due to the structure of the questionnaire. One way this happens is due to the manner in which options for a multiple-choice (or nominal) question are sequenced. The options at the top tend to be selected more than those in the middle, or at the end. This is especially true when the list of options is long.

How does one overcome this issue? By randomizing the options. Option randomization removes the fixed order of options and sends them out in a randomized sequence. Now different options would show up in different options to each respondent. By doing this, it removes the bias due to the position of the option.

In Merren, you can randomize the options by just toggling to “Randomize Options” (1) when setting up a nominal question.

Sometimes you would want to fix the last option and not randomize it. In the above example, the last option is “Other”. You might want to keep this fixed to the last position and not be randomized. For that you can toggle off randomization for the last option (2). Do note that if you have only two options and you switch randomization off for the last (second) option, no randomization would happen.


When not to randomize?

If your list of options is ordered i.e. the sequence in which the options are presented is meaningful, you should not randomize your options. For example, you are collecting the age range and the options are less than 18 years, 18-30 years, 31-40 years, 41-50 years and so on. In this case, the sequence of options is meaningfully ordered (ascending order). In this case, you should not randomize the choices