Determine the Survey Length with Merren Method

Let’s start with the why: Here are the good, bad and the ugly reasons why the length of a survey gets inflated:

The Good

• 1. Decision making is complicated and requires collecting data on multiple metrics.
• 2. Pressure testing the critical data is important. You need corroborating evidence to support your decision making. So you need to collect that data as well.
• 3. Collecting a certain type of data enables you to take certain action. Example: Net Promoter Score (NPS) survey. NPS asks brand advocacy based on a single question.

“On a scale of 0-10, how likely are you to recommend <BRAND NAME> to your friends/ colleagues”

If you use this technique to improve your product/ service, you would want to know the reasons behind every rating response. Without a follow up question (such as an open ended question), you will not get the data to diagnose any future issues.

The Bad

Lack of comprehension of the objectives – here is a quote that we fancy as stated by Blaise Pascal: “If I had more time, I would have written a shorter letter.” The market research equivalent of it would be, “If I had spent more time on understanding the decision, I would have written a shorter survey.”

When a business is unsure of its market research goals, they can get tempted to cover multiple metrics of measurements. This causes ambiguous market research. Clarity is an underrated tool. It helps cut through the fluff and seek feedback on the meaningful metrics.

The Ugly

1. The spray and pray method:

Sometimes, you may understand the survey objectives. Unfortunately, market researchers may not put enough effort into understanding the metrics needed to measure and understand objectives. So you apply the spray and pray strategy where you frantically collect information of all sorts, hoping that some answer will help you make a decision.

This bloats your survey design and hurts the quality of data since survey respondents also tire out.

2. Survey fatigue:

It is tough to convince a customer to answer a long survey. It can get tedious, time consuming and expensive. When a respondent agrees for a survey, the greed sets in: “let us ask them everything we possibly can.” This can feel like an interrogation.

3. “While you are at it” syndrome:

In larger companies, you would often see requests to increase questionnaire length from an unrelated department as soon as it is known that a survey is being conducted.

So how do you optimize the length of your survey? Seasoned market researchers use the following framework to categorize each question:

1. Critical: Must have a number of questions that will help you with the decision making. The threshold to enter the category should be high. If you see this list going beyond 4-5 information areas, you are probably not doing it right.
2. Support: Questions that validate and verify the findings of your critical questions.
3. Good to know: Questions that would be helpful to know and understand, but do not directly pertain to the decision at hand.
4. Everything else: Usually everything that comes after “Well, if we are doing a customer survey, maybe we can also…”

Every question in your questionnaire should be put through the filter and categories. Once you are sure of the survey length, you should retain all As, rationalize Bs, actively reduce Cs and eliminate Ds. The question format and framework, when done well, forces you to cut down on the survey fat. One should also consider what will be the amount of time taken to answer the survey. This brings us to survey completion time for both short surveys and long surveys. 

What is Survey Completion Time? 

Survey completion time is the time taken to complete a survey. Short and long surveys are as follows: 

Short surveys vs. long surveys

Both short surveys and long surveys serve different purposes. Choosing the right format depends on the objectives:

• Short Surveys (Under 5 minutes): Best for quick feedback on a rating scale, in-app surveys, NPS, or post-purchase experiences.
• Long Surveys (Over 10 minutes): Suitable for detailed market research, employee engagement studies, demographic studies and in-depth customer insights.

Make your long surveys engaging without losing data quality:

• Use skip logic/ branching to hide irrelevant questions. It is also helpful to personalize questions based on previous responses. It is commonly used in Net Promoter Score surveys. 
• Prioritize must-know questions; remove “nice-to-have” items. Combine similar questions and avoid redundancy.
• Ensure a clear and engaging layout to reduce cognitive load.
• Offer progress indicators to keep respondents motivated.
• Provide survey response incentives to encourage participation.

Strategies to increase survey completion rate

1. Choose the right survey channel: Interactive survey channels have a greater chance of completion rate. Interactive survey channels include native WhatsApp surveys, messenger or chatbot surveys. WhatsApp’s 98% open rate encourages higher completion rate. Chatbot’s interactive human-like conversation encourages people to offer responses.  

2. State survey intention upfront: Mention the reason for the survey, how you will use the information and how long it will take. This will allow customers to anticipate and share responses without uncertainty. 

Design Tip: Include a progress bar to Show respondents how many questions are left

3. Offer survey incentives and rewards: Even small survey rewards as much as $10 can boost completions, especially for longer surveys. Ensure that the rewards are appropriate as per the product, industry and target respondents. 

4. Offer in-moment surveys: In-moment surveys are questions that are published immediately during or after an interaction. It can post an interaction, purchase or experience to boost response rate. The interaction is relevant for the customer. They are likely to offer authentic and genuine responses. 

Factors Influencing Survey Length

Several factors play a critical role in determining the appropriate survey length, each tailored to the context of the research:

Target respondent demographics:

• Audiences such as Gen Z who extensively use social media platforms may not answer long surveys. A segment of respondents may also have shorter attention spans. They prefer concise surveys. Demographics influence patience levels, with younger groups favoring brevity.
• General research trends have noted that senior demographics (60 years and above) may be willing to invest time to provide detailed responses.

Purpose of the survey:

• For quick feedback or routine checks, shorter surveys are ideal. They are particularly effective for mobile-first designs where brevity and its interactivity encourages completion rates.
• Longer surveys are suitable for in-depth research or topics of significant interest to respondents, such as longitudinal studies or highly engaged customer bases. While the golden standard is 10-12 minutes, longer surveys can be acceptable if the topic requires depth (if questionnaires are engaging).

Complexity of questions:

• Simple, single-choice questions can be answered quickly. Estimates suggest an average of 7.5 seconds per question. However, complex questions, such as open-ended or grid-based ones, require more time and cognitive effort, potentially extending the survey duration.
• This complexity can lead to respondent fatigue, especially in longer surveys, impacting data quality.

Conclusion

Create customer feedback surveys with Merren, an AI-powered customer feedback platform. Build surveys in a few clicks and publish it across multiple survey channels. Interactive and ready-to-use for any CX professional. Sign up for a 14 day free trial here.