How to design a questionnaire for needs assessments in ...

July 2016

Technical Brief

How to design a questionnaire for needs assessments in humanitarian emergencies

? Sandie Walton-Ellery, 2012

ACAPS Technical Brief ? Questionnaire Design

ACKNOWLEDGEMENTS This document would not have been possible without the assistance, support and guidance of many people. ACAPS especially wishes to express its gratitude to Aldo Benini, Emanuele Bruni, Sandie WaltonEllery, Richard Garfield, Assanke Koedam, Edouard Legoupil, Benoit Munsch, Leonie Tax, Lynn Yoshikawa and Melissa Weihmayer.

2

Questionnaire Design Checklist

ACAPS Technical Brief ? Questionnaire Design

This checklist outlines the essential components of a questionnaire design process. Each of the topics below is discussed in detail in subsequent sections.

PLANNING

ASSESSMENT OBJECTIVES and INFORMATION NEEDS have been agreed by all stakeholders There is a comprehensive and detailed ANALYSIS FRAMEWORK and PLAN TESTING, TRAINING and, if necessary, TRANSLATION have been included in the planning

FORMAT AND CONTENT

Questionnaire format is EASY TO USE The questionnaire starts with METADATA and CLASSIFICATION questions The FLOW and ORDER of questions are effective General questions before specific questions Questions are ordered in terms of importance Questions on behaviour (what people do) come before questions on attitude (how people feel) Spontaneous, open questions before prompted, multiple-choice questions Sensitive questions are placed at the end of the questionnaire Transition statements are included for each new topic, timeframe, or unit of measurement The key informant (KI) or household (HH) questionnaire is in most instances no longer than 50

MINUTES (20 for telephone surveys), focus group discussions (FGDs), and community group discussions (CGDs) no longer than 90 MINUTES Clear INSTRUCTIONS are provided on how to ask and answer every question Questions and sections are VISUALLY DISTINCT. There is sufficient SPACE for the enumerator to record answers

QUESTION QUALITY

Key questions have been TESTED and IMPROVED based on the feedback OPEN-ENDED QUESTIONS are only used if really necessary The questions are UNDERSTANDABLE Questions are stated in a simple, straightforward manner Every question has a clear unit of measurement, e.g. change `how long' into `how many hours' There are no ambiguous terms or timeframes The question stem and choices match Every question asks about one concept only Questions are ANSWERABLE Questions have been adapted to the knowledge and language of the respondent Skip questions (questions that are included or excluded based on the respondents' answers) are

used where necessary and the skip instructions are correct) Response categories include `do not know' and `no response' The response categories are mutually exclusive, as exhaustive as possible, as precise as necessary,

and meaningful to respondents The denominator is clear in questions asking for percentages or ratios Questions are UNBIASED Bias through social desirability is minimised, meaning that questions are not presented in a way that

might lead a respondent to think one response is preferred over another Sensitive questions are carefully introduced and phrased

3

ACAPS Technical Brief ? Questionnaire Design

Content List

......................................................................................................................................................................................... 1 Questionnaire Design Checklist ................................................................................................................................3 Content List...................................................................................................................................................................4 Introduction ................................................................................................................................................................... 5 I - Purpose and principles...........................................................................................................................................5

Remember the objectives ......................................................................................................................................5 Apply a user-centred design ..................................................................................................................................6 Minimise the risk of error .......................................................................................................................................6 Adhere to ethical standards...................................................................................................................................7 II - Questionnaire Modes ............................................................................................................................................7 III - Designing a questionnaire ...................................................................................................................................9 Step 1: Identify objectives and information needs ......................................................................................... 10 Step 2: Decide on source of information and data collection technique ................................................... 10 Step 3: Draft questionnaire(s) ............................................................................................................................ 12 Step 4: Review feasibility ..................................................................................................................................... 13 Step 5: Finalise the analysis plan ....................................................................................................................... 13 Step 6: Structure and format .............................................................................................................................. 15 Step 7: Translate ................................................................................................................................................... 16 Step 8: Field test and adapt ................................................................................................................................ 17 Step 9: Instruct field teams ................................................................................................................................. 18 Step 10: Review the questionnaire .................................................................................................................... 19 IV- Designing good questions................................................................................................................................. 20 Decide the level of measurement ...................................................................................................................... 20 Decide on types of questions ............................................................................................................................. 20 Designing closed-ended questions ................................................................................................................... 21 Designing open-ended questions ...................................................................................................................... 22 Question wording.................................................................................................................................................. 23 Annex A: Prioritising questions in emergencies.................................................................................................. 28 Example prioritisation criteria ............................................................................................................................ 29 Annex B. Debriefing form ........................................................................................................................................ 30 Annex C: Metadata and classification questions ............................................................................................... 31 References ................................................................................................................................................................. 31

4

Introduction

There is no shortage of questionnaires used during emergencies that are too long, overly complex or unable to generate useful responses. The art of developing an effective questionnaire is the topic of master degrees and doctorates. The questionnaire is a critical tool in humanitarian response and requires time, resources and a detailed understanding of the context, factors which are all in short supply during an emergency. This technical brief aims to support the design of questionnaires for use in humanitarian emergencies by providing a set of guiding principles and a step-by-step process. If many of the principles detailed are universal to the development of any type of questionnaire, most of the examples used in this brief are related to strategic needs assessments implemented at the community level rather than the household level, e.g. Multi Cluster Initial and Rapid Assessments.

Considering the complexities involved in developing a reliable data collection tool, we emphasize the need to develop the questionnaire and test it during the assessment preparedness phase. Dedicating time and resources during "peace time" goes a long way in avoiding common mistakes in questionnaire design.

The brief starts with an explanation of the main purpose of a questionnaire and the principles that should be followed to reach these objectives. Afterwards, the ten steps of questionnaire development are discussed. The brief concludes with sections on what to keep in mind specifically when designing a questionnaire and individual questions. It focuses on questionnaire design for intervieweradministered (as opposed to self-administered) surveys, as this is the most common approach used in humanitarian emergencies. However, Section two briefly touches upon the impact of different survey modes on questionnaire format and wording.

This brief focusses on how to gather the information required; it does not provide recommendations on what information should be collected. In addition, the design, roll-out and ultimately the success of an assessment involves much more than designing a questionnaire. It includes deciding the sampling strategy, arranging logistics, data processing, etc. This brief touches on these other components only as far as they are relevant to the design of questionnaires. For more information on the complete assessment process and which information to gather when, see The Good Enough Guide ? Humanitarian Needs Assessment (2014) and the MIRA revision July 2015.

ACAPS Technical Brief - Questionnaire Design

I - Purpose and principles

Questionnaires are specialised and structured tools of human interaction. They are meant to make communication more effective and predictable. The main objective of a questionnaire is to translate what actors need to know (their information needs) into a set of questions that respondents are able and willing to answer. More specifically, a questionnaire aims to :

Ask the right question, in the right way, to the right

person.

Provide a structure to the interview that enables it

to proceed smoothly and systematically.

Systematise responses and the forms on which

these are recorded to facilitate data processing and, most importantly, analysis.

To achieve these objectives, four principles are essential in designing a questionnaire:

Remember the objectives Apply a user-centred design Minimise the risk of error Adhere to ethical standards

Remember the objectives

Keep in mind the assessment objectives at every stage of the assessment process. Problems encountered during questionnaire development often stem from a lack of clarity on the assessment objectives. A questionnaire that fails to translate the objectives clearly is inevitably going to overlook important issues and waste resources and participants' time by asking irrelevant questions. Strict adherence to what the assessment intends to measure, and what it does not, helps keep the questionnaire focused.

Determining the purpose of the information collected is one of the best ways to clarify the specific goals of an assessment. It is all too common to include questions without evaluating their contribution towards the assessment objectives. Irrelevant questions are costly for those conducting the assessment and for those responding to the questionnaire. Therefore, do not include a question unless the data it provides can be of direct use in addressing the objectives. The exception to this rule are questions included to establish rapport between the enumerator and respondent, or bridgebuilding questions which reorient the respondent's perspective in preparation for the next section of the questionnaire.

5

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download