ࡱ> DGABC bjbj|| =Z+Z+88888T8889}:8!=:[=@===???$I@:8Av?@?AA:88==OHHHA|8=8=HAHH^=@8?Bthe0+D++8p?v,@TH@D@???::gF???AAAA+?????????Z+ c7:   Focus Group Interviewing Research Methods PRIVATE  University of Minnesota Notes Richard A. Krueger, Ph.D. and Mary Anne Casey, Ph.D. University of Minnesota HYPERLINK "mailto:rkrueger@umn.edu"rkrueger@umn.edu HYPERLINK "mailto:casey016@umn.edu"casey016@umn.edu September 2015 Table of Contents OVERVIEW Characteristics of focus group interviews.. 2 MODERATING Moderator skills.. 3 Example of a focus group introduction .. 4 Outline of a focus group introduction . 5 Moderator checklist ...6 Suggestions for creating a discussion....7 Assistant moderator role....8 Note taking...... 9 How to give an oral summary at the end of the focus group .11 DEVELOPING QUESTIONS Good questions..12 Active questions that engage the participants..13 Generic questions Consumer product... 14 Generic questions Program evaluation..15 PLANNING Planning the focus group study..16 Developing a focus group plan...17 RECRUITMENT Systematic notification procedures, selection, incentives...18 ANALYSIS & REPORTING Analysis choices19 Tips for transcribing focus group interviews.20 Systematic analysis process......21 Focus group analysis tips.......22 Steps in the analysis process 23 Classic analysis strategy....24 Reporting focus group results.......25 OTHER Telephone focus groups.....27 Internet focus groups......28 BIBLIOGRAPHY Bibliography .... 29 Characteristics of Focus Group Interviews Participants 5 to 8 people per group Carefully recruited Have screens in commonhomogeneity Repeated groups with different participants Environment Comfortable for participants Circle seating Audio recorded Moderator Skillful in group discussions Uses pre-determined questions Establishes permissive and nonthreatening environment Analysis and Reporting Systematic analysis Verifiable procedures Appropriate reporting Meditation Seeing all Beings as Enlightened Picture or imagine that this earth is filled with Buddhas, that every single being you encounter is enlightened, except one--yourself! Imagine that they are all here to teach you. Whoever you encounter is acting as they do solely for your benefit, to provide just the teachings and difficulties you need in order to awaken. Sense what lessons they offer to you. Inwardly thank them for this. Throughout a day or a week continue to develop the image of enlightened teachers all around you. Notice how it changes your whole perspective on life. --A Path with Heart-- Jack Kornfield Moderator Skills Select the right moderator team Moderator and assistant moderator Establishes rapport Shows interest Moderator Introduces study Guides discussion Assistant moderator Handles logistics Takes careful notes Listens for what more is needed Asks final question(s) Gives summary May be primary analyst Be mentally prepared Alert and free from distractions Ready to listen Understands purpose and questioning route Act as hosts Create warm and friendly environment Use purposeful small talk Observe the participants for seating arrangements Make a smooth & snappy introduction Standard introduction 1. Welcome 2. Overview of topic 3. Ground rules 4. First question  Use pauses and probes 5 second pause Probes: "Say more." "Would you give an example?" "I don't understand." Record the discussion Digital recorders Written notes Control reactions to participants Verbal and nonverbal Head nodding Short verbal responses (Avoid "that's good, excellent") Use subtle group control Experts Dominant talkers Shy participants Ramblers Use appropriate conclusion Four Step Conclusion 1. Asst moderator asks 1 or 2 questions 2. Asst moderator summarizes key points and asks for confirmation 3. Review purpose and ask if anything has been missed 4. Thanks and dismissal Example of a Focus Group Introduction The first few moments of a focus group are critical. In a brief time, the moderator must create a thoughtful, permissive atmosphere, provide ground rules, and set the tone of the discussion. Much of the success of group interviewing can be attributed to the development of this open environment. The recommended pattern for introducing the group discussion includes a: (1) Welcome, (2) Overview of the topic, (3) Ground rules, and (4) First question. Here is an example: Good evening and welcome. Thanks for taking the time to join us to talk about health and wellness in your community. My name is Dick Krueger and with me is Tom Olson. We're both with the University of Minnesota. Sarah Casey, who is with the county public health office, asked us to get some information from community residents about health and wellness. The public health office is developing a new strategic plan to decide what priorities they should have over the next five years. They want your input about health issues, what worries people here in the community, and what could be done to improve the health of people who live in the county. Your input, along with input from other groups being held across the county, will be used to set priorities for the coming years. You were invited because you live here in the community and you have had a chance to listen to what people in the community say about health and wellness. There are no wrong answers, only differing points of view. Please feel free to share your point of view even if it differs from what others have said. Keep in mind that we're just as interested in negative comments as positive comments, and at times the negative comments are the most helpful. You've probably noticed the recorder here. We're recording the session because we don't want to miss any of your comments. People say really helpful things in these discussions and we can't write fast enough to get them all down. We will be on a first name basis tonight, and we won't use any names in our reports. We will keep what you say confidential. Our reports will go back to the county public health staff to help them plan future programs but, again, no names will be attached. Ive got a number of questions that I want to ask, but my job is really to listen. This will be more interesting for all of us if we treat this like a conversation. If someone says something, feel free to follow up on it or share a different point of view. You dont need to address all your comments to me. If you have a cell phone, please put it on silent mode. If you need to take a call, please step out and then return as quickly as possible. Let's begin. We've placed your first names on cards in front of you to help us remember each other's names. Let's find out more about each other by going around the table. Tell us your name and then tell us what you like best about living in this community. Outline of a Focus Group Introduction Welcome Introduce moderator and assistant The purpose ... What prompted the study? Who asked for it? (Who is the sponsor?) What will be done with the information? (How will the findings be used?) Benefits of the study (How will it help participants, community members, etc.?) Why you were invited ... You were invited because (You have characteristics, expertise, or experiences that can help us) Guidelines (Adapt to group) Were audio recording We're on a first name basis. No names attached. It will help if only one person talks at a time. If several of you are talking at the same time, the recording gets garbled and well miss your comments. No wrong answers, only differing points of view. You don't need to agree with others, but you must listen respectfully as others share their views Please silence your phones. If you must respond to a call, please do so as quietly as possible and rejoin us as quickly as you can. The bathroom is down the hall. We are going to be informal here tonight, so feel free to get up for more coffee or refreshments. My role is to guide the discussion Talk to each other (If you know participants are talkative) I may need to interrupt to get through all the questions. I apologize ahead of time if I need to do this. Opening question Moderator Checklist I. Before the focus group Understand goal of the sponsor and purpose of study. Understand the purpose of each question. Have a sense of the amount of time needed for each question. Anticipate topics of discussion and potential areas of probing. Be mentally and physically ready to moderate. Have sufficient technical knowledge of topic. Practice using recording device. II. During the focus group Establish rapport with participants. Convey sense of relaxed informality. Act as hostwelcome participants and make them feel welcome. Deliver introduction with enough information to put participants at ease, including: A welcome A brief overview that defines the purpose for the group Ground rules (or things that will help our conversation go better) The opening question Keep discussion on track and keep participants involved. Get people to talk. Listen carefully. Probe for clarification. Seek out both cognitive and affective domains. Get participants to tell both how they think and how they feel about the topic. Handle different participants adeptly. Bring closure to the group with a summary and invite comments on any missing points. Go to the door and thank each person individually for coming, just as you would when guests leave your home. III. After the focus group Debrief soon after the focus group with assistant moderator. Review transcript before next focus group to determine how to improve discussion. Perform the analysis or provide insight into the analysis. Review the report for accuracy. Suggestions for Creating a Discussion Instead of a Serial Interview Begin with a permissive and non-threatening environment Be sure participants can all see each other Tell them you would like a discussion Review your questions for conversational potential Are the questions intriguing? Do questions raise your curiosity? Do answers quickly come to mind? Do answers of others interest you? Do participants have experience with the topic? Use strategies to encourage conversation Ask the question and look at someone After someone has spoken, wait for another to talk Ask, What else? Let your eyes linger on someone Keep looking as they answer (if they stay on topic) Watch for participants who might want to talk Look at someone else as if you want that person to talk Use body gestures to signal encouragement to speak Bring others into the conversation Are there comments from anyone else? Call on someone who has not spoken and ask for their comment. Alicia, you havent had a chance to say anything. Be ready for follow-up questions, such as: "How did that make you feel?" "What was that like?" "How did that happen?" "What led up to that?" "Now what?" or "What happens next? Assistant Moderator Role Help with equipment & refreshments Help arrange the room Welcome participants as they arrive Help with any paperwork at registration table Sit outside the conversation circle Draw diagram of where people are sitting Take notes throughout the discussion Do not participate in the discussion Think about what needs amplification Answer questions (as topic expert) when asked Ask questions when invited Give an oral summary Give incentive if there is one Thank each person for participating Debrief with moderator Give feedback on analysis and reports Note Taking Note taking is a primary responsibility of the assistant moderator The moderator should not be expected to take written notes during the discussion. Clarity and consistency of note taking Anticipate that others will use your field notes. Sometimes an analyst might not look at field notes for days or weeks following the focus group when memory has faded. Consistency and clarity are essential. Field notes contain different types of information. Know what the analyst wants. Your field notes may contain: Identification information Include the date, time, location of the focus group. Name of moderator and assistant. Types of participants. Diagram of table with names. Quotes Listen for notable quotes, the well-said statements that illustrate an important point of view. Listen for sentences or phrases that are particularly enlightening or eloquently express a particular point of view. Place name or initials of speaker after the quotations. Usually, it is impossible to capture the entire quote. Capture as much as you can with attention to the key phrases. Use three periods ... to indicate that part of the quote is missing. Key points and themes for each question Typically, participants will talk about several key points in response to each question. These points are often identified by several different participants. Sometimes they are said only once but in a manner that deserves attention. At the end of the focus group, the assistant moderator will share these themes with participants for confirmation. Follow-up questions that could be asked Sometimes the moderator may not follow-up on an important point or seek an example of a vague but critical point. The assistant moderator may wish to follow-up with these questions at the end of the focus group. Big ideas, hunches, or thoughts of the recorder Keep track of your ideas, hunches, and thoughtsbut circle them as a way of separating them from quotes. These insights are helpful in later analysis. Names Sometimes it is important to attach names to transcripts. If so, the assistant moderator might write down the name and the first few words each time someone talks. Later, these notes can be merged with the transcript. Other factors Make note of factors that might aid analysis such as passionate comments, body language, or non-verbal activity. Watch for head nods, physical excitement, eye contact between certain participants, or other clues that would indicate level of agreement, support, or interest. How to Give an Oral Summary at the End of the Focus Group Before the focus group, be sure you know the key questions and the approximate time the moderator plans to spend on each key question. Be clear about the purpose of the focus group. The summary should tie closely to this purpose. Take two kinds of notes: a) Notes that will help you provide a brief oral summary, and, b) Notes for detailed analysis after the focus group. Make sure to leave enough time for the summary. If you give the summary two minutes before you said the discussion would be over, participants may be eager to leave and may not offer feedback. Begin your oral summary with the most important findings regardless of when they were discussed in the focus group. Don't worry about the question sequence when you construct your summary. Begin your summary with findingswhat was actually said. Attempt to capture common themes but also acknowledge differing points of view. This descriptive summary repeats what was said but is very brief. After you've given the summary of what was said, consider offering interpretation. The interpretative summary attaches additional meaning and goes beyond the actual words. Listen for what was not said, but might have been expected. If these areas are important, then in the summary you might say: "Some things were not mentioned like . . . . Are they important to you or not?" Look at the participants while you're saying this and watch for reactions. Cite key phrases used in the discussion. This demonstrates connectedness and careful listening. Keep the summary to three minutes or less. If you ramble, people will tune out. When finished, look at the participants and ask, "What would you add to that summary?" Or, "Does that reflect the conversation you heard?" Or, "What would you change about this summary?" Good Questions Sound conversational Use words participants would use Are easy to say Are clear and short Are open-ended Are one-dimensional Include good directions Tips on Developing Questions Avoid dichotomous questions Why? is rarely asked Use "think back" questions Use different types of questions Opening Question Introductory Question Transition Questions Key Questions Ending Questions Include an ending question If you had one minute to give advice to (name of person in charge), what would you say? "Of all the things we discussed, what to you is the most important?" "Have we missed anything?" Be cautious of serendipitous questions Keep most questions consistent to allow for comparison between different types of audiences (e.g., teachers, students, parents) Use a team to help develop questions Anticipate several revisions Active Questions Engage the Participants Make a list, record results on flip chart, then discuss Draw a picture Side-by-side comparison Draw diagrams, flowchart, etc Role play or demonstrate Group sorting or assembly Complete a rating sheet and discuss Walk around the museum for 15 20 minutes and then discuss Observe something (e.g., video, demonstration) and then discuss Construct something (e.g., collage, toy) and then discuss Doing something before the focus group Keep a log Go somewhere and observe Make a scrapbook Talk to friends or family Look at a website Review materials Generic Questions Consumer Product Purpose: Better understand what features consumers look for in a product Participants: People who make decisions to buy this productQuestionsMinutes1.How and when do you use ..... ?52.Tell me about positive experiences you've had with ....103.Tell me about disappointments you've had with ....104.Who or what influences your decision to purchase .... ?105.When you decide to purchase .... , what do you look for? Take a piece of paper and jot down three things that are important to you when you purchase .....106.Let's list these on the flip chart. If you had to pick only one factor that was most important to you, what would it be? You can pick something that you mentioned or something that was said by others.107.X is most important to a number of you. Talk about that.158.Y is most important to others. Talk about that.159.Have you ever changed brands or types of ..... (Raise hands) What brought about the change?1010.If you had one minute to give advice to the people who design , what would you say?10105 Program Evaluation Purpose: To gain input on strengths of the program and what could be improved. Participants: Adults who have been in the program at least two years.QuestionsMinutes1.How did you get involved in (name of program)? 52.Think back over all the years that you've participated in (name of program) and tell us your fondest memory. 103.Lets talk about strengths. a. Take this piece of paper and write down three things (name of program) does particularly well. (Give time to write.) b. OK. Lets go around the table and each of you tell me what you wrote down and give me a one-sentence description of that thing. (List each item on a flip chart. If an item is mentioned more than one time, put a check mark next to it for each additional time it is mentioned.) c. (Pick the one with the most check marks and say) A number of you said X was a strength. Talk more about that. (Discuss two or three itemsas time allows.)254.Now lets talk about what needs improvement. Lets use the same process. a. Using the same piece of paper, write down three things (name of program) could improve. b. OK. Lets go around the table and each of you tell me what you wrote down and give me a one-sentence description of that thing. (List each item on a flip chart. If an item is mentioned more than one time, put a check mark next to it for each additional time it is mentioned.) c. (Pick the one with the most check marks and say) A number of you said X could be improved. Talk about that. What is the problem and how could it be improved? (Discuss two or three thingsas time allows.)255.Suppose that you were in charge and could do whatever you wanted to improve the program better. What change would you make? 106.What can participants do to make the program better? 107.One of the problems that the staff struggle with is declining participation. How could they get more people to participate?158.If you invited a friend to participate in the program, what would you say? 109.What to you is the most important thing weve discussed? 10120 Planning the Focus Group Studies 1. Decide whether focus groups are appropriate 2. Decide whom to involve (What makes someone information-rich for this study?) 3. Get advice from a few people who are information-rich 4. Put your thoughts in writing Planning Guide for Focus Groups Assuming a focus group study of 4-6 focus groups within the same community. 1. Planning Conceptualizing the study, developing questions and arranging logistics Time Needed: 6-70 hours 2. Recruiting Developing recruitment strategy and scripts, contacting potential participants, follow-up with letters and phone messages. Time Needed: 15-50 hours 3. Moderating Moderate focus groups, plus travel time for moderator and assistant. Time Needed: 24-36 hours 4. Analysis Analyze data and prepare written report. Time Needed: 8-120 hours But: Add 35% more time if you've never done it before Add 20% more time if a committee has to approve draft Add 20% more time if recommendations are needed 5. Other Costs: Travel expenses for moderator team, travel expenses for participants, incentives, food, room charge, childcare, transcription charges, digital recording and transcription equipment, supplies like flip charts, markers, art supplies. Developing a Focus Group Plan A written plan will help you organize the proposed study in a logical manner and allow others to provide feedback. Consider these ingredients. The Focus Group Plan1. Name of the study 2. Purpose / Research question 3. Participants 4. Recruitment procedure & incentive 5. Number of groups & locations 6. Resources needed (time and money) 7. Timeline  Systematic Notification Procedure IMPORTANT: This process is personalized and repetitive 1. Set meeting times for group interviews 2. Contact potential participants via phone or in person 3. Send a personalized confirmation letter 4. Phone (or contact) each person the day before the focus group General Selection Rules 1. Set exact specifications 2. Maintain control of selection process 3. Use the resources of the sponsoring organization in recruiting 4. Beware of bias participants picked by memory participants picked because they've expressed concern participants who are clones of the person doing the selection participants who are unproductive 5. Develop a pool of eligible participants and then randomly select 6. Balance cost and quality Selection Strategies Usually more effective Lists Nominations Work with local organizations Piggyback On location Usually less effective Snowball samples Random telephone screening Screening and selection services Ads in newspapers and bulletin boards Incentives for Participation Money/Gift cards Food Gifts Positive, upbeat invitation Opportunity to share opinions Opportunity to hear what others have to say Enjoyable, convenient and easy to find meeting location Involvement in an important research project Build on existing community, social or personal relationship Analysis Choices PRIVATE ANALYSIS TYPEMemory- based analysisNote-based analysisAbridged transcript analysisTranscript- based analysis DESCRIPTIONModerator analyzes based on memory and past experiences Gives oral debriefing & PowerPoint to clientModerator prepares a brief written description based on summary comments, field notes and selective review of recording Moderator prepares written report based on an abridged transcript after listening to audio plus field notes and moderator debriefingAnalyst prepares written report based on complete transcripts. Some use of field notes and moderator debriefing ORAL OR WRITTEN REPORTSUsually oral report & PPUsually oral and written reportUsually oral and written reportUsually oral and written reportTIME REQUIRED PER GROUPVery fast 1 hour per group Fast About 1-3 hours per groupSlower About one day per group Slow About 2 days per groupPERCEIVED LEVEL OF RIGORMinimalModerateModerate to HighHighRISK OF ERRORHighModerate-depending on quality of field notesLowLow Tips for Transcribing Focus Group Interviews Use quality playback equipment A transcription kit with a foot pedal allows you to control the playback speed without removing your fingers from the keyboard. Check to see if your software has noise-reduction features that minimize the background sounds on recordings. Minimize distractions Type transcripts at a time and place with few distractions or interruptions. Use a consistent style Include name of study, date and location of focus group, type of participants, number of participants, name of moderator and assistant moderator. Identify moderator comments using a consistent style such as bolding, capitalization or underlining. We prefer bolding. Single space all comments. Double space between speakers. Dont type verbal fillers such as umm, ahh, or you know Number all pages. Place a header on all pages indicating date and group name. Don't worry about punctuation People don't speak in complete sentences. Use punctuation where it seems to make sense. Place periods at what seem to be the end of sentences. Type comments word for word This is a transcript, not minutes of a meeting. The conversation is transcribed word for word. If someone repeats something that was previously said, type it again. Don't change the words or correct the grammar. If some of the words are unintelligible then insert [unintelligible] to indicate that words are missing from the transcript. Note special or unusual sounds that could help analysis Indicate things like laughter, loud voices, shouting, someone being interrupted. Put these inside brackets [laughter]. Allow sufficient time Typically, it takes about four to eight hours to type a two-hour focus group. But the time will vary with typist speed, the quality of the recording, the length of the session, the experience of the typist with focus groups, and the complexity of the topic. Do it yourself One of the best ways to improve your moderating skills is to prepare your own transcript. This helps you improve your future moderating and helps you in the analysis. For greatest benefit, prepare the transcript soon after the focus group is completed. Decide whether to attach names to comments This is difficult to do and usually isnt done because it is so time consuming. The typist usually needs to merge the transcript with the field notes of the assistant moderatorand the field notes must be taken in a way that facilitates identification of the speakers. Systematic Analysis Process 1. Start while still in the group Listen for inconsistent comments and probe for understanding Listen for vague or cryptic comments and probe for understanding Consider asking each participant a final preference question Offer a summary of key questions and seek confirmation Draw a diagram of seating arrangement 2. Immediately after the focus group Check the audio recording to ensure you captured the discussion Conduct debriefing Note themes, hunches, interpretations, and ideas Compare and contrast this focus group to other groups Label and file field notes, tapes and other materials 3. Soon after the focus group--within hours analyze individual focus group Make backup copy of recording Transcribe the focus group (if using transcripts) Idealmoderator/asst moderator transcribes and reviews before next focus groupthis improves the quality of data gathered in next group. Look for places where moderating can be improved and where additional data are needed. (Think of what the client needs.) If not using transcripts, moderator listens to recording paying attention to how to improve moderating and where additional data are needed. 4. Later--within days analyze the series of focus groups Compare and contrast results across focus groups. Compare all like-groups first. (For example, compare teenager groups then compare parent groups.) Look for emerging themes by question and then overall Construct typologies or diagram the analysis Describe findings and use quotes to illustrate 5. Finally, prepare the report Consider narrative style versus bulleted style Use a few quotes to illustrate Sequence could be question by question or by theme Share report for verification with other researchers Revise and finalize report Focus Group Analysis Tips When analyzing focus group data consider . . . WORDS Think about both the actual words used by the participants and the meanings of those words. A variety of words and phrases will be used to describe issues and the analyst will need to determine the degree of similarity between these responses. CONTEXT Responses are triggered by a stimulusa question asked by the moderator or a comment from another participant. Find the triggering stimulus and then interpret the comment in that context. Interpret responses in light of the preceding discussion and by the tone and intensity of the oral comment. INTERNAL CONSISTENCY Sometimes participants in focus groups change or even reverse their positions after hearing what others have said. This phenomenon rarely occurs in individual interviews due to a lack of interaction with others. When there is a shift in opinion, the researcher typically traces the flow of the conversation to determine clues that might explain the change. FREQUENCY OR EXTENSIVENESS Extensiveness refers to the number of participants who mentioned a topic. Frequency if the number of times a topic is mentioned. Dont confuse these. One topic could be brought up a lot, but only by one participant. Frequency and extensiveness are factors to consider when the analyst is deciding what weight to give a topic. INTENSITY Look for emotion. Pay attention when participants talk about a topic with a special intensity or depth of feeling. Intensity may be difficult to spot from transcripts because it is communicated by the voice tone, speed, emphasis on certain words, and body language. Individuals will differ on how they display strength of feeling and for some it will be a speed or excitement in the voice whereas others will speak slowly and deliberately. SPECIFICITY Give more weight to responses that are specific and based on experiences than responses that are vague and impersonal. To what degree can the respondent provide details when asked a follow-up probe? Give more attention to responses that are in the first person as opposed to hypothetical third person answers. For example, "I feel the new practice is important because I have used it for three months and had great results," has more weight than, "These practices are good and others in the area should use them." FINDING BIG IDEAS One of the traps of analysis is getting caught up in the details. Step back from the discussions. Allow time for big ideas to percolate. For example, after finishing the analysis the researcher might set the report aside for a brief period and then jot down the three or four of the most important findings. Assistant moderators or others skilled in qualitative analysis might review the process and verify the big ideas. Steps in the Analysis Process 1. Decide who will coordinate the analysis 2. Look over your questions and specifically think about analysis 3. Anticipate the discussion 4. Use multiple strategies for data capture 5. Work with note taker to ensure the right data are captured 6. Some analytic tasks occur during the focus group 7. Debrief soon after the group 8. List, file, copy and document all materials 9. Decision: Should you transcribe? 10. Begin the coding process 11. Prioritizing the analytic themes 12. Write the report The Classic Analysis Strategy: Long Tables, Scissors and Colored Marking Pens Equipment needed: Two copies of all transcripts Scissors Tape Lots of room with long tables and possibly chart stands Large sheets of paper (flip charts, newsprint paper, etc.) Colored marking pens Stick-on notes 1. Prepare your transcripts for analysis. You will save time and agony later if you are careful when preparing your transcripts. Be sure they follow a consistent style. For example, single space comments and double space between speakers. Bold, cap, or underline the comments of the moderator so they are easy to see. 2. Make two paper copies of each transcript. You will cut one up, and use the other for later reference. TIP: Consider printing transcripts on different colors of paper and color coding by audience type, category, etc. For example, print teenagers on blue paper and parents on green paper. Or draw one blue line down the right-hand margin of each page of the first teenager group, and two down the margin of each page of the second teenager group. After the transcripts are cut apart it will be easy to tell that any comment with two blue lines came from the second teenage group. 3. Arrange transcripts in an order. It could be in the sequence in which the groups were conducted, but more likely it will be by categories of participants or by some demographic screening characteristics of participants (users, non-users and employees, or teens, young adults and older adults, etc.). This arrangement helps you be alert to changes that may be occurring from one group to another. 4. Read all transcripts at one sitting. This quick reading is just to remind you of the whole scope and to refresh your memory of where information is located, what information is missing, and what information occurs in abundance. 5. Prepare large sheets of paper. Use one large sheet of paper for each question. Place the large sheets on chart stands, on a long table or even on the floor. Write the question at the top of the sheet. If you have several categories of groups you might draw lines to divide the paper into sections and then group comments within these sections. For example, on one part of the page you might place comments from teen focus groups, in another place there will be comments from parent focus groups, and in a third place there will be comments from teacher focus groups. 6. Cut and tape. Start with the first question you are analyzing. (You might not analyze the first question.) a. Cut out and read the first response. Ask yourself: Does this answer the question? If yes, start a pile. If no, does it answer another question? If yes, move it to that question. If no, save it in a pile to review later. b. Cut out and read the second response to the question. Does it answer the question? If yes, is the answer similar to the first answer? If so, put it in the same pile. If it isnt similar, make a new pile. If no, does it answer another question? If yes, move it to that question. If no, save it in a pile to review later. You are constantly comparing new information to what was already said and putting like things together. c. Continue cutting and comparing responses to the first question until you have completed all the transcripts. d. Go to the next question. e. When you have completed all the questions, go back to the first question and review the various piles that you have. Now you may think that some of these piles are related and should be combined. Or you may realize that a pile contains several different concepts and you want to tease those apart. Rearrange your piles or categories until you feel they adequately convey what people said in the groups. f. Tape each pile or category to the paper. Give each pile a label or descriptor. 7. Write a statement about the question. Look over the quotes and write a paragraph that describes the responses to that question. A number of possibilities may occur. For example, you might be able to compare and contrast differing categories, you might have a major theme and a minor theme, you might discuss the variability of the comments, or even the passion or intensity of the comments. Following the overview paragraph, you may need several additional paragraphs describing sub-themes or to elaborate on selected topics. When you are finished, go on to the next question. 8. Continue until all transcripts are reviewed. 9. Take a break. Get away from the process for a while. Refocus on the big picture. Think about what prompted the study. It's easy to get sidetracked into areas of minor importance. Be open to alternative views. Be skeptical. Look over the pile of unused quotes. Think big picture. Invite a research colleague to look over your work and offer feedback. 10. Prepare the report. Reporting Focus Group Results Use a communications strategy Rather than thinking of "a report," think of what type of communication strategy is needed. A variety of reports might be used to keep people informed. Consider: e-mails, postcards, phone calls, bulleted summaries, selected quotes, moderator comments, mid-project or final project reports, personal visits by members of the research team, etc. Never surprise a client with bad news As evaluators, sometimes we learn that not all is well with a program or service. Do no surprise clients with this at a final meeting. Communicate problems early. Use an appropriate reporting style that the client finds helpful and meets expectations Ask users what kind of report would be helpful to them. What information are they looking for? What are the expectations and traditions of reports within the organization? Ask for examples of styles of reports that they have found useful. Strive for enlightenment Reports should raise the clients level of understanding. The purpose is more to enlighten and convey new insights as opposed to repeating what is already known by the sponsor of the study. Make points memorable Help clients remember the key points by limiting the number of points you highlight. Too many points diminish overall impact. Begin with most important points and follow with lesser important points. Use narrative or bulleted format Written reports can follow either a narrative format or a bulleted format. Don't surprise the client with a format different from what was expected. Give thought to the oral report Oral reports should be brief, clear, and concise. Presenters should indicate why the study is important, why the findings are meaningful, begin with the most important findings, engage the listener in an active manner, and allow plenty of time for questions. Telephone Focus Groups Guiding Principles of Telephone Focus Groups Small size group 4 to 5 people recommended Share questions or discussion topics in advance Limit questions to 5 to 8 Limit the length to about 60 minutes Have participants identify themselves (if voices are not recognizable) Encourage participants NOT to use speaker phones or cell phones (poor sound) Call on people who arent actively participating Periodically ask for round-robin responses Include ending questions Suggestions for a Successful Telephone Focus Group Work with a colleague who can help you: Keep track of who is talking and not talking Note topics that are not adequately discussed Take notes and capture relevant quotes Typically this person doesnt speak, but is in the room with the moderator and communicates via notes It is OK to list items before they speak or use a simple rating scale Be watchful for participants who get stuck on minor items and be ready to move them along to the next question. At the end, you might indicate that if they have any additional thoughts that werent covered, or if something occurs to them later, that you would appreciate their email comments. Internet Focus Groups Internet groups show enormous potential, but technical glitches remain a problem One concern is the platform or computer system that handles the groups. Market researchers contract this out to technical firms. Often public, non-profit and academic communities dont have the budget for this out-sourcing and therefore, try to adapt in-house systems (such as Moodle, Adobe Connect, or forum features in other systems.) Be sure your systems are password protected Keep everything simple and easy to use Have tech support ready to help when problems emerge Test out the system and questions several times before beginning the first group Consider using avatars (pseudonyms) to provide confidentiality instead of their real names Use back-channel communications to encourage discussion. Back-channel is when you send email messages to individuals encouraging them to add their comment, or to comment in more detail, or to thank them for their thoughtful comment. This lets them know that you are paying attention. The Chat Room Focus Group The chat room focus group consists of a limited number of people who sign on to a designated Internet site at a particular time and then a moderator presents a series of questions to the participants. These synchronous sessions are often limited to 6 - 8 participants and last 90 minutes or less. These chat room groups tend to elicit the top-of-mind answers and emphasize speed over thoughtful response. They offer the excitement of a live exchange but it is also an environment where some participants can dominate the conversation. The Bulletin Board Focus Group In the bulletin board focus group, a limited number of people agree in advance to participate in an asynchronous electronic discussion over the course of several days. They agree to sign on each day and check the bulletin board, read the question(s) for the day, look over the comments of other participants, and formulate their response. Often participants are asked to spend about 20 to 30 minutes per day for several days and incentives are based on how many days they submit responses. The questions in a bulletin board group should have a focus and move in a general direction over the course of several days. Bibliography Bystedt, J., Lynn, S., & Potts, D. (2003). Moderating to the max. Ithaca, NY: Paramount Market Publishing. Guest, G., Bunce, A., and Johnson, L. (2006). How many interviews are enough?: An experiment with data saturation and variability. Field Methods, 2006, 18;59. Grob, G.F. (2010). Writing for impact. In J.S. Wholey, H.P. Hatry, & K.E. Newcomer. (Eds.). Handbook of practical program evaluation. Chapter 25, pp 594 619. Hennink, M. M. (2007). International focus group research: A handbook for the health and social sciences. New York: Cambridge University Press. Houston, T.K., et al. (2011). Culturally appropriate storytelling to improve blood pressure. Annals of Internal Medicine. 154(2) 77-84. Krueger, R. A. & Casey, M. A. (2015). Focus groups: A practical guide for applied research. 5th edition. Thousand Oaks, CA: Sage. Krueger, R.A. (1998). Developing questions for focus groups. Moderating focus groups. Analyzing and reporting focus group results. Thousand Oaks, CA: Sage. Krueger, R. A., & King, J. A. (1998). Involving community members in focus groups. Thousand Oaks, CA: Sage. Merton, R., Fiske, M., & Kendall, P. L. (1956). The focused interview. Glencoe, IL: Free Press. Morgan, D. L. (1997). Focus groups as qualitative research. Thousand Oaks, CA: Sage. Morgan, D. L., & Krueger, R. A. (Eds.). (1998). The focus group kit. Thousand Oaks, CA: Sage. Patton, M. Q. (2014). Qualitative evaluation and research methods (4th ed.). Thousand Oaks, CA: Sage. Rosser, B.R.S., et al. (2011). The future of Internet-based HIV prevention: A report on key findings from the mens INTernet (MINTS-I, II) sex studies. NIH Public Access. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3345812/ Vaughn, S., Schummn, J. S., & Sinagub, J. (1996). Focus group interviews in education and psychology. Thousand Oaks, CA: Sage. qualcore.com QualCore.com Inc. is a firm providing traditional and Internet-based qualitative research services. It is located in Minneapolis, MN. HYPERLINK "http://www.itracks.com"www.itracks.com itracks is a company that specializes in providing bulletin board services for market researchers. It is located in Saskatoon, SK, Canada. Revised 9.6.15     PAGE  PAGE 1 Focus Group Interviewing --- Richard A. Krueger & Mary Anne Casey 2015 PAGE  Focus Group Research Methods -- R.A. Krueger & M.A. Casey 2014 Focus Group Research Methods -- R. A. Krueger & M.A. Casey 2014 Focus Group Research Methods -- R. A. Krueger & M. A. Casey 2015  PAGE \* MERGEFORMAT 7  +2345=>?IMVWX[]cefxǽֶ|sjs|b^bh%>jh%>Uh]5CJ^JhXF5CJ^Jh~]h]5CJ^Jh]5CJOJQJ h]5CJ0 h]5CJ hu|5CJ h0^5CJ hJ5CJ h]5CJ h]5CJ$\jh%>5CJ$U\h]5CJ$\ hN 5CJH h]5CJHjh%>UhmHnHuh] "+34?WXYZ[\]cdef$ 0*$]a$gd] 0*$gd]*$gd]! " # 2 3 4 5 6 7 I J U ^gdJ^gddgdJgdj$a$gdjgd]$ Hpdh*$^pa$gd]$ 0*$]a$gd]   ! # , - 0 1 4 5 6 7 8 A I J L U ^ ¹~o~bXNAh0hJOJQJ^JhJOJQJ^Jh=OJQJ^Jh0hjOJQJ^Jhj5CJ$OJQJ^JaJ$#h0hj5CJ$OJQJ^JaJ$h] h]5CJ$hnh]5CJ^Jh]5CJ^Jh>Y}5CJ^Jh"o5CJ^Jh~]h]5CJ^Jh%> hXFh]5B*CJ^Jphjh%>U'hXFh]0J5>*B*CJ^Jph^    . B Z [ p 8 > ? T c |  ! " I [ y  ) V b ~ ߷hN OJQJ^Jh0hdOJQJ^Jh/FxhJOJQJ^JaJhJOJQJ^JaJh0hJOJQJ^JhJOJQJ^JhdOJQJ^JB / [ ? @ U } ! " + \  * c gdJ^gdd h^h`gdJ^gdJ *^_e ;<=J  d^ gdjgdj d^gdjgdj$ H6*$a$gdj^gdL ^gdJgdJ^gdJ ()G\]^_{  =J;=[]ȿhehj5CJaJ hehjhjOJQJ h =\ hj\h = hj5hb@5 hjCJhjhj5CJaJh0hL OJQJ^JhJOJQJ^JhdOJQJ^Jh0hJOJQJ^JhX4OJQJ^J1Jbv;[ & F 0 *$]^ gdj  ^ gdj & F 0  *$]^ gdj ^ gdj 02 *$]^2 gdj & F 0  *$]^ gdj#hiE\l 06*$gdj H6*$gdj$ H6*$a$gdjgdj  H *$^ gdj & F 0 *$]^ gdj  d^ gdj 02 *$]^2 gdj#$hilm*p+Pj3JNA[z|} ;!#ֻⲬۥ✖ hpT ^JhpT hpT ^J hXF5CJ$ hjCJ$hj5CJ$\ hj5<hb@5 h&*hj hXF5hXF hj5 hj5CJ$hjhehjaJ hXFaJhehj5aJ9lm~*<Pp^gdb@5 $ H*$a$gdjgdj+Pj3BJVsAgdjAB[ct,bz{|gdpT 0*$gdj $ H*$a$gdjd1$gdj+,SU>A>B>úͳͺͭͭͭͭͧͧ h6q^J h62+^J h62+5^Jh62+h62+^Jh62+h62+5^Jh62+hpT ^Jh62+hpT 5^Jh]CJOJQJ h]5CJ$ h =CJhJah]CJA677B788888'9(9/9::;|<}<<z={== ^`gdpT ^gdpT 8^8gd62+j^jgd62+ & F.8^8gdpT gd62+ & F-gdpT gdpT ^gdpT =C>D>E>F>L>&?'?5?G@H@I@J@g@@@A & FgdW>c 0d0*$gdW>c$ Hd0*$a$gdW>c $d0a$gdW>cgd] & F.8^8gdpT ^gdpT ^gd62+ & F/gd62+^gd62+^gdpT B>C>D>F>K>L>&?'?4?G@H@I@J@K@@@AtAAAAHDLDFFGGGĺį~vrnrnrirnr]NhVlh]CJOJQJaJh6qCJOJQJaJ hW>c>*h&hW>chW>cOJQJ hW>c5CJ$hW>c5CJ$\hW>cCJOJQJaJh]5CJOJQJh]CJOJQJhpT 5CJOJQJh62+hpT 5^Jh62+hpT ^JhN hh62+5CJOJQJh62+5CJOJQJh6qCJOJQJh62+CJOJQJ hpT ^JAAqArABBBBCCEEFFFFFFGGG $ H1$a$gd] 00*$^`0gd] & F 0d0*$gdW>c 0d0*$gdW>cGGGG6H8HTHUHVHWHsHtHuHHHHHHHHHHII'I(I8I9IEIJJJ*J,J1JJJbJJJJJJJJú庱xۺooooooh6qh6qaJh6qh]CJaJhh6qh]5CJaJh6qh]CJaJh6qhU^CJaJh6qh =CJaJh6qhU^aJh6qh =aJhN hN 5CJ$aJ$hN h]5CJ$aJ$h6qh]5aJh6qh\u,aJh6qh]aJh]5CJ$\+GGGGG HH6H7H8HUHsHHHHH 0v *$^v gd6q & F 0dh*$^gd\u, & F dh^gd t & F 0dh*$^gd t $ H*$a$gdN $ H*$a$gd] & F,  H dh*$^ gd =HHII'I(IEIJJ*JJJJxh 0*$^gd6q & F 0*$^gd6q & F 0dh*$^gd t gd]  ^gd =  & F gd t 0@ *$]^@ gd] 0v *$]^v gd6q v 1$^v gd6q v d^v gd6q JJJJJJKKKWKXKgKhKpKKKKKKKKKKKKLL6L7LX@XAXDXnXXXXXXXXXYYYYYYYYYYYY6Z:Z;Z $$IflFh $JJ t0    44 layt:NUUUUUVVYE33 0d $*$Ifgd  0d @$*$Ifgd kd $$IflFh $JJ t0    44 layt:N$ 0d @$*$Ifa$gdJaVVnW=X@XAXDXG3 0d @$*$Ifgd kd $$IflFh $JJ t0    44 layt:N$ 0d @$*$Ifa$gdJa 0d $*$Ifgd DXXXXXG3 0d @$*$Ifgd kd? $$IflFh $JJ t0    44 layt:N$ 0d @$*$Ifa$gdJa 0d $*$Ifgd$XYYYYYG3 0d @$*$Ifgd kd $$IflFh $JJ t0    44 layt:N$ 0d @$*$Ifa$gdJa 0d $*$Ifgd YYYYYYYE3 0d $*$Ifgd  0d @$*$Ifgd kd $$IflFh $JJ t0    44 layt:N$ 0d @$*$Ifa$gdJaYYY7Z:Zp\J3$ 0d @$*$Ifa$gdJa 0d $*$Ifgd  0d @$*$Ifgd kd@$$IflFh $JJ t0    44 layt:N:Z;ZcCJaJh$CJaJhnth]CJaJhh]5CJ$aJ$h]5CJ$aJ$hE h]5 h]CJ hU^CJ hU^5CJ$h] h]5CJ$2P[Q[R[S[[[[\J\c\d\s\\ ] ]]_]y]z]]]]]^ 06*$gdE 06*$`0gdE H6*$gd] 06*$gd]^<^n^o^^h___``-`$$If^a$gd H*$gd] $ H*$a$gd] 006*$^`0gd] 06*$gd] -`.`C`D`E`F` $If^gd jkdA$$Ifl$ & t0644 la^yt F`G`f`g`h`i`j`ttttt $If^gd }kd$$Ifl0~ $ ~ t0644 la^yt\u,j`k`{`|`}`~```tttttt $If^gd }kde$$Ifl0~ $ ~ t0644 la^yt\u,```````````ttttttttt $If^gd }kd$$Ifl0~ $ ~ t0644 la^yt\u, ``````````tttttttt $If^gd }kd$$Ifl0~ $ ~ t0644 la^yt\u, ````````aaaaa8aJaeapaqavaaaaaaaabbbb>bAbPbbbb_d`dadwdddd_e`e~effffٔjh%>Uh+h+^Jh+h+>*^Jh+h]5aJh&h&^J h&5CJ$h+h$hJah\u, h]>*h~h]>*hnr h]5CJ$ h\u,5CJ$ h\u,h$h] h\u,h\u, h\u,h]1``````aaattttttt $If^gd }kd<$$Ifl0~ $ ~ t0644 la^yt\u,aaaaaaattttt $If^gd }kd$$Ifl0~ $ ~ t0644 la^yt\u,aaa8aaab?bbttfXXXX 0*$^gd& 08*$^8gdnr $ H*$a$gd]}kdv$$Ifl0~ $ ~ t0644 la^yt\u,bbbbbb8cJchccccBd_d`dadwdddddddd$a$gd+ H*$gd] & F0gd&`gd& Hgd& 0*$gd]ddde9e_e`e~eeeeeef9ffffff 06*$gd]$ 06*$a$gd] & F h0  *$^ gd+ & F h0  *$^ gd t H*$gd] 0*$gd]$a$gd+ffffffffff gg g0g2g=gugwggghhi*i+iCiiiiiii6jNjwjjjjjk?k@kkktkllillllmmmmmmmm n n[nɻhU^CJ^JaJh$CJ^JaJh$h]CJ^JaJh$h]5CJ^JaJh]5OJQJ h]5CJ$h+ h; `5 hJa5h] h]5jh%>U h]gghhiv`J```$ 0Z6$*$Ifa$gd+$ 0Z6$*$Ifa$gd kd$$Ifxr?$PPP44 xaxyt ii+iDidiiiv`````$ 0Z6$*$Ifa$gd kd$$Ifxr?$PPP44 xaxyt iiiiiiijjv````L`L$ 06$*$Ifa$gd $ 0Z6$*$Ifa$gd kd$$Ifxr?$PPP44 xaxyt jj5j6jOjWj`jqjvjLkd\$$Ifxr?$PPP44 xaxyt $ 06$*$Ifa$gd $ 0Z6$*$Ifa$gd vjwjjjjjjv`````$ 0Z6$*$Ifa$gd kd$$Ifxr?$PPP44 xaxyt jjjjjjkvj]]SI 0*$gd] H*$gd] $ H*$a$gd] 0l*$gd]kd$$Ifxr?$PPP44 xaxyt kllliljllmmmm nZn[n{n o o)o{p|pp,q-qDqHr & F# 0*$^gd t 0*$gd] 0*$gd][nxnzn o'o|pp-qBqHrIrOrWrXrwrssVsWssssss4t9tttttttttuuuٿٿٰ٦ْٜنwrkfaf]fah] hU^5 h]5 hd)Phd)P h]hhVlh]CJOJQJaJh]CJOJQJaJhiCJ^JaJh\u,CJ^JaJh]CJ^JaJh$h]CJOJQJaJh$CJ^JaJh$h]5CJOJQJaJh$h]CJ^JaJh$h]5CJ^JaJhi5CJ^JaJ$HrIrXrUsVssttttttuSuuuuuv 0| *$^`| gd] & F h088*$^8gd]gdd)P1$gd] 00*$^`0gd] 00*$`0gd] 0*$gd]uvw w wQwwwwwZx{x y*y4y7y8ymyyyzzzz{{{{{||~~~ɄrsuxhCJ^JaJhb*4CJ^JaJhhb*4>*CJ^JaJhhb*45CJ^JaJhhb*4CJ^JaJhhb*4CJ^Jhhb*45CJ^J h]5CJ h]5CJ$h+hihYbh]5 hg{Ih] hU^5h] h]5/vUvhvvvwwRwqwwx3y4yny & F 0*$gd t & F h088*$^8gd] & F h088*$^8gd] 0*$gd] 0 *$` gd] & F 0*$gd t & F h088*$^8gd] nyz8zezzzzz{6{k{{{{{{gdb*4 H6*$gd]$ H6*$a$gd] & F h088*$^8gd] 0*$gd] 0*$^`gd] & F h088*$^8gd] & F h088*$^8gd]{|||~~~}~ʄstuvx†Æ1$gdX4 $1$a$gdX4gdb*4uvx†$%RSLJȇ>?\]É.norڻ짣r$hb*4hb*45^Jehr hb*4^Jhb*4hb*45^Jhb*4hb*4^J hb*4hh] h]5CJ$hX4CJPJ^JaJhh'hX4CJPJ^JaJhhX45CJ$PJ^JaJ$h"h'hX45CJ$PJ^JaJ$h hX45CJ$hhCJ^JaJ)Æ%&STȇɇ?@]^ 0d*$gd] Hd *$gd]$ Hd *$a$gd]1$gdX4&+c‰ÉmnLM܎ݎŏƏ5 00$d%d&d'd*$NOPQ^`0gdb*4 0*$gdb*4 & F1 0*$gdb*41$gdb*4rƌMoݎƏה۔xKdQRhopq.ӝ*4 ՠʢh]5CJ aJ h]5h{h]5OJQJh]5OJQJh]OJQJh] h]5CJ$ hb*45hb*4 hb*4^Jhb*4h5^J hb*45^Jhb*4hb*45^Jhb*4hb*4^J!hb*4hb*4^Jehr0Ərߒ)S"lo%wx & F 0d*$gdb*4 & F 0d*$gdb*4 0hd*$^hgdb*4 0d*$gdb*4 0*$gdb*4QRklmnq.ҝӝ 06*$^gd] 006*$^`0gd] 06*$gd] $ 0*$a$gd] 0*$gdN 0d*$gd] 0*$gdb*4 0d*$gdb*4ӝ+5 Ԡՠʢˢ'Xs & F 0*$1$gd t 0*$gd]$a$gd] 006*$^`0gd] 06*$gd]ʢˢ'(XYpst/0abڤۤܤ1756SZȸxhb*4CJOJQJaJhKkCJOJQJaJh)Lh]CJOJQJaJhU^CJOJQJaJh]5CJOJQJaJh)Lh]5CJOJQJaJh]CJOJQJhU^CJOJQJhKkh]h$hU^ h]5CJh~h]5CJ aJ /s/aۤ1_5^_`acy$a$gd]^gd] & F)gdi & F(gdigd]^gd] & Fgd t & F 0*$1$gd t^`acyz|ͧΧЧ'+"#%QRT{|~ efhܭ?oïį0DFGH̸̸̸̸̸̸̸̸̸̸̼̫̟̒̒̒̊ h]5CJ$h]OJQJhX;hKkh]^JhDh]5CJaJh]5CJaJhthKkh|h_Mh]5CJ aJ h]h]CJOJQJh]5CJOJQJh)Lh]CJOJQJaJhb*4CJOJQJaJ3yzͧΧ"#QR{|efܭE$a$gd]gdKk 0P(6*$]P^(gdKk & F& 0P6*$]P^gd tgd]EFGUV°ab<= `gdhgd ` HPP6*$]P^Pgd\u, 0PP6*$]P^Pgd]gd]HUVǰְװ 9DERSTYZ]_`ԴԴveQ@Q hX;6CJOJQJ]^JaJ&h `ha6CJOJQJ]^JaJ h:6CJOJQJ]^JaJhhCJOJQJ\^JaJ#h `haCJOJQJ\^JaJhX;CJOJQJ\^JaJh:CJOJQJ^JaJhX;CJOJQJ^JaJ#h `ha6CJOJQJ^JaJ h `haCJOJQJ^JaJhhCJOJQJ^JaJh `h]5CJ$aJ$`abjsl;>@ByŷŷtbbTbhvECJOJQJ^JaJ#h `ha6CJOJQJ^JaJ h `h> CJOJQJ^JaJ#hzhz6CJOJQJ^JaJ#hzhX;6CJOJQJ^JaJh:CJOJQJ^JaJhX;CJOJQJ^JaJhzCJOJQJ^JaJh> CJOJQJ^JaJhaCJOJQJ^JaJ h `haCJOJQJ^JaJyz|ܳݳ!"bڴ!Fѵӵյ !&'=˹˹˜˹˹˹ˎ˹rdS h `h:CJOJQJ^JaJhaCJOJQJ^JaJh `CJOJQJ^JaJhuCJOJQJ^JaJhvECJOJQJ^JaJhGCJOJQJ^JaJh:6CJOJQJ^JaJ#h `ha6CJOJQJ^JaJ h `haCJOJQJ^JaJ&hvEhvE6CJH*OJQJ^JaJhvE6CJOJQJ^JaJ `a'(  gdl $`a$gdvE`gd:`gdh=>@G ;o !CDSTbָ򙋙k\J\J\#h `haCJOJQJ\^JaJh:CJOJQJ\^JaJ'h `ha0JCJOJQJ\^JaJh%>jh%>Uh|CJOJQJ^JaJ h `haCJOJQJ^JaJ$h `ha0JCJOJQJ^JaJ#h `h:6CJOJQJ^JaJ#h `h:CJOJQJ\^JaJ h `h:CJOJQJ^JaJh:CJOJQJ^JaJ H_Ĺȹιعÿwhwhwh$56CJOJQJaJ"hnh$56CJOJQJaJh>Y} hh$h>Y}0JmHnHuh$ h$0Jjh$0JUhf'#jhf'#UhvEhuCJOJQJ\^JaJhYCJOJQJ\^JaJhvECJOJQJ\^JaJhCJOJQJ\^JaJ( 9-m$h$d%d&d'dNOPQ]ha$gd oo$a$oh]h o&`#$gd mh]hgd mh]h m&`#$gd ع "/089:;<X\]^_jxy˿ߩˡˡˇ˃hvEhf'# hwh$jh$U*hzomHnHujh$U hh$h>Y}56CJOJQJaJ h$0J h$CJ h$h^RJh$6CJOJQJaJ"hnh$56CJOJQJaJh$56CJOJQJaJ*9:;< $`a$gdvEoh]hgd o$a$mh]hgd -m$h$d%d&d'dNOPQ]ha$gd mh&`#$]hgd d ; 000:pABP/ =!"#$%` X 0 000&P:pABP/ =!"#$%` P0 21h:pA/ =!"#$% 9 01h:pA/ =!"#$% P 5 01h:pA/ =!"#$% {$$If!vh5h%#vh%:V l t065h%yt $$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa$$If!vh5.5<5#v.#v<#v:V l t065.5<5ytJa}$$If!vh5h%#vh%:V l t05h%yt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If!vh55J5J#v#vJ#vJ:V l t055J5Jyt:N$$If^!vh5 &#v &:V l t065 &a^yt $$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$If^!vh5 5~#v #v~:V l t065 5~a^yt\u,$$Ifx!vh555P5P5P#v#v#vP:V x555P/ /  / / 4 xaxyt $$Ifx!vh555P5P5P#v#v#vP:V x555P/ / / / 4 xaxyt $$Ifx!vh555P5P5P#v#v#vP:V x555P/ / / / 4 xaxyt $$Ifx!vh555P5P5P#v#v#vP:V x555P/ / / / 4 xaxyt $$Ifx!vh555P5P5P#v#v#vP:V x555P/ / / / 4 xaxyt $$Ifx!vh555P5P5P#v#v#vP:V x555P/ / / /  / 4 xaxyt n 6666666~~~666666666666666666666666666666666666666666666H666666666666666666666666666666666666666666666666666666666666666662 0@P`p2( 0@P`p 0@P`p 0@P`p 0@P`p 0@P`p 0@P`p8XV~$OJPJQJ^J_HmH nH sH tH L`L ]Normal1$CJPJ^J_HhmH sH tH b`b ] Heading 1!$0*$1$@&^`0CJhmHsHtHX`X ] Heading 2$ F0*$@&5CJmHsHtHf`f ] Heading 3$$dh0*$1$@&]^5CJhmHsHtH``` ] Heading 4$0*$1$@&]^Z5CJhmHsHtHN`N ] Heading 5 $@&a$5CJ \mHsHtH^`^ ] Heading 6$H*$1$@&CJOJQJhmHsHtH^`^ ] Heading 7$dh0*$@&^@ 5CJmHsHtH\`\ ] Heading 8$H*$1$@&a$5CJ$hmHsHtH\ `\ ] Heading 9 $0*$1$@&]5CJhmHsHtHDA`D Default Paragraph FontRi@R 0 Table Normal4 l4a (k ( 0No List @/@ ]Heading 1 Char PJ^JaJF/F ]Heading 2 Char5PJ^JaJhD/D ]Heading 3 Char5PJ^JaJD/!D ]Heading 4 Char5PJ^JaJN/1N ]Heading 5 Char5CJ PJ\^JaJhH/AH ]Heading 6 CharOJPJQJ^JaJJ/QJ ]Heading 7 Char5CJPJ^JaJhH/aH ]Heading 8 Char5CJ$PJ^JaJH/qH ]Heading 9 Char5CJPJ^JaJP+`P ] Endnote Text 0*$CJmHsHtHJ/J ]Endnote Text CharPJ^JaJhT* T ]Endnote ReferenceCJH*OJQJmH sH uR R ] Footnote Text 0*$CJmHsHtHL/L ]Footnote Text CharPJ^JaJhV& V ]Footnote ReferenceCJH*OJQJmH sH u,, ] Document 86/6 ] Document 4 56CJ,, ] Document 6,, ] Document 5D/!D ] Document 2CJOJQJmH sH u,1, ] Document 7.A. ] Bibliogrphy.Q. ] Right Par 1.a. ] Right Par 2D/qD ] Document 3CJOJQJmH sH u.. ] Right Par 3.. ] Right Par 4.. ] Right Par 5.. ] Right Par 6.. ] Right Par 7.. ] Right Par 8d/d ] Document 1.$$ 0*$1$CJPJ^J_HhmH sH tH (( ]Doc InitB/B ] Tech InitCJOJQJmH sH u.. ] Technical 5.!. ] Technical 6F/1F ] Technical 2CJOJQJmH sH uF/AF ] Technical 3CJOJQJmH sH u.Q. ] Technical 4F/aF ] Technical 1CJOJQJmH sH u.q. ] Technical 7.. ] Technical 8@@ ]Default Paragraph Fo:: ]_Equation Caption.. ] Bullet List.. ] Document[8]8/8 ] Document[4] 56CJ.. ] Document[6].. ] Document[5]F/F ] Document[2]CJOJQJmH sH u.. ] Document[7]0!0 ] Right Par[1]010 ] Right Par[2]F/AF ] Document[3]CJOJQJmH sH u0Q0 ] Right Par[3]0a0 ] Right Par[4]0q0 ] Right Par[5]00 ] Right Par[6]00 ] Right Par[7]00 ] Right Par[8]f/f ] Document[1]K$$ 0*$1$CJPJ^J_HhmH sH tH 00 ] Technical[5]00 ] Technical[6]H/H ] Technical[2]CJOJQJmH sH uH/H ] Technical[3]CJOJQJmH sH u00 ] Technical[4]H/H ] Technical[1]CJOJQJmH sH u0!0 ] Technical[7]010 ] Technical[8]f/Bf ] Default ParaT 0*$1$#OJPJQJ^J_HhmH sH tH .Q. ] Document 8a8/a8 ] Document 4a 56CJ.q. ] Document 6a.. ] Document 5a.. ] Document 2a.. ] Document 7ah/h ] Right Par 1a[ 0*$1$CJPJ^J_HhmH sH tH l/l ] Right Par 2a\ 0*$1$CJPJ^J_HhmH sH tH .. ] Document 3an/n ] Right Par 3a^ 0p*$1$CJPJ^J_HhmH sH tH r/r ] Right Par 4a_ 0p@ *$1$CJPJ^J_HhmH sH tH t/t ] Right Par 5a"` 0p@ *$1$CJPJ^J_HhmH sH tH x/x ] Right Par 6a%a 0p@ *$1$CJPJ^J_HhmH sH tH z/"z ] Right Par 7a(b  0p@ *$1$CJPJ^J_HhmH sH tH ~/2~ ] Right Par 8a+c 0p@ *$1$CJPJ^J_HhmH sH tH f/Bf ] Document 1ad$$ 0*$1$CJPJ^J_HhmH sH tH d/Rd ] Technical 5ae 0*$1$"5CJPJ^J_HhmH sH tH d/bd ] Technical 6af 0*$1$"5CJPJ^J_HhmH sH tH 0q0 ] Technical 2a00 ] Technical 3ad/d ] Technical 4ai 0*$1$"5CJPJ^J_HhmH sH tH 00 ] Technical 1ad/d ] Technical 7ak 0*$1$"5CJPJ^J_HhmH sH tH d/d ] Technical 8al 0*$1$"5CJPJ^J_HhmH sH tH J`J n]Headerm !*$CJmHsHtH>/> m] Header CharPJ^JaJhJ `J p]0Footero !*$CJmHsHtH>/> o]0 Footer CharPJ^JaJhBB ]pTOC 1q$ *$]^`0>> ]pTOC 2r$ *$]^`0>> ]pTOC 3s$ *$]^p`0>> ]TOC 4t$ *$]^@ `0>> ]TOC 5u$ *$]^`0:: ]TOC 6v$*$^`022 ]TOC 7 w*$^`0:: ]TOC 8x$*$^`0:: ]TOC 9y$ *$^`0B B ]Index 1z$ *$]^``B B ]Index 2{$ *$]^`0>.> ] TOA Heading |$*$*"* ]Caption}</< ]_Equation Caption1PT@P ] Block Text 0*$1$]^ph.)@. ] Page NumbertC`t ]Body Text Indent"d00*$^`0CJOJQJmHsHtHZ/!Z ]Body Text Indent CharOJPJQJ^JaJhPZ`2P ] Plain Text1$CJOJQJhmHsHtHN/AN ]Plain Text CharCJOJPJQJ^JaJ6U`Q6 ]0 Hyperlink >*B*ph*W a* ]Strong5\FV qF ]FollowedHyperlink >*B* ph ] Table Grid7:V0 dCJOJPJQJ^JaJV^V ]0 Normal (Web)dd1$[$\$OJQJaJh<@< ] List Paragraph^``` ] No Spacing 1$`$CJOJPJQJ^J_HmH sH tH   > p TOC Heading*$d 0*$@& ^`'5B*CJOJPJQJ\^JaJph6_P P > 0 Balloon TextCJOJQJaJmHsHtHV/V > 0Balloon Text CharCJOJPJQJ^JaJhPK![Content_Types].xmlj0Eжr(΢Iw},-j4 wP-t#bΙ{UTU^hd}㨫)*1P' ^W0)T9<l#$yi};~@(Hu* Dנz/0ǰ $ X3aZ,D0j~3߶b~i>3\`?/[G\!-Rk.sԻ..a濭?PK!֧6 _rels/.relsj0 }Q%v/C/}(h"O = C?hv=Ʌ%[xp{۵_Pѣ<1H0ORBdJE4b$q_6LR7`0̞O,En7Lib/SeеPK!kytheme/theme/themeManager.xml M @}w7c(EbˮCAǠҟ7՛K Y, e.|,H,lxɴIsQ}#Ր ֵ+!,^$j=GW)E+& 8PK!Ptheme/theme/theme1.xmlYOo6w toc'vuر-MniP@I}úama[إ4:lЯGRX^6؊>$ !)O^rC$y@/yH*񄴽)޵߻UDb`}"qۋJחX^)I`nEp)liV[]1M<OP6r=zgbIguSebORD۫qu gZo~ٺlAplxpT0+[}`jzAV2Fi@qv֬5\|ʜ̭NleXdsjcs7f W+Ն7`g ȘJj|h(KD- dXiJ؇(x$( :;˹! I_TS 1?E??ZBΪmU/?~xY'y5g&΋/ɋ>GMGeD3Vq%'#q$8K)fw9:ĵ x}rxwr:\TZaG*y8IjbRc|XŻǿI u3KGnD1NIBs RuK>V.EL+M2#'fi ~V vl{u8zH *:(W☕ ~JTe\O*tHGHY}KNP*ݾ˦TѼ9/#A7qZ$*c?qUnwN%Oi4 =3ڗP 1Pm \\9Mؓ2aD];Yt\[x]}Wr|]g- eW )6-rCSj id DЇAΜIqbJ#x꺃 6k#ASh&ʌt(Q%p%m&]caSl=X\P1Mh9MVdDAaVB[݈fJíP|8 քAV^f Hn- "d>znNJ ة>b&2vKyϼD:,AGm\nziÙ.uχYC6OMf3or$5NHT[XF64T,ќM0E)`#5XY`פ;%1U٥m;R>QD DcpU'&LE/pm%]8firS4d 7y\`JnίI R3U~7+׸#m qBiDi*L69mY&iHE=(K&N!V.KeLDĕ{D vEꦚdeNƟe(MN9ߜR6&3(a/DUz<{ˊYȳV)9Z[4^n5!J?Q3eBoCM m<.vpIYfZY_p[=al-Y}Nc͙ŋ4vfavl'SA8|*u{-ߟ0%M07%<ҍPK! ѐ'theme/theme/_rels/themeManager.xml.relsM 0wooӺ&݈Э5 6?$Q ,.aic21h:qm@RN;d`o7gK(M&$R(.1r'JЊT8V"AȻHu}|$b{P8g/]QAsم(#L[PK-![Content_Types].xmlPK-!֧6 +_rels/.relsPK-!kytheme/theme/themeManager.xmlPK-!Ptheme/theme/theme1.xmlPK-! ѐ' theme/theme/_rels/themeManager.xml.relsPK] 56 {##=##ˠ# HHH^ #)/0a4B>GJLRWZ`f[nuurʢH`y=ع^abehmptux{} JlAo$\'})+n/}1<46=AGHJLN,NRNNN OO|PPPWQQQgR{RR[SUVVDXXYY:ZAZP[^-`F`j````aabdf1giijvjjkHrvny{ÆƏӝsyE 9_`cdfgijklnoqrsvwyz|~ CSXXX  !!!!L# @~(  n  S  C"?n  S  C"?B S  ?h%%@8h@$$@8h@_D _ _ _&_,&_l&_&_&==Ka!]GTTc!]=*urn:schemas-microsoft-com:office:smarttags PlaceName=*urn:schemas-microsoft-com:office:smarttags PlaceType9*urn:schemas-microsoft-com:office:smarttagsplace - ~CZ:>ceIWy~%G[{:=I <ARaNT).V:DU[lq*:=EBGs33333333333333333333333333333333333333333333+2MV#,UU}()`b()\] m'n'T@T@r~u~v~׌یoo>@yѭӭ H_+2MV#,UU}()`b()\] m'n'T@T@r~u~v~׌یoo>@yѭӭ H_1q!(:4>,l B3 l   2>>00 *do`!%KS!#NdUNM$ cT( E;~+Ps+$/"'/w/JƋ+1%nF~8QN9 4{ho;Jgm2h;BY",< 4{A=,+V>ذ0y> 4{/2C >C 4{D }'vH^[ _b bL Wc"à.cl Vh<1ހxh"GC/DNM$n`O ohRcT(]+>[DxN}>N9<KS!#HL+1(E;~+Vh+V>'ufoxh,l ho;IxMPy>s+F~8k~2 .cxA=}'vH2h;2b/w/o`!Wcq11                                                                                                                                                                                   ~p        ~p                                                                               V3j>pT N _ > L $uh+.v'f'#")62+\u,b*4b@5X;=%>V!Fy+J M~M:Nd)P72P]0^ `; `W>c(m"ozox&rnr/Fx${u|>Y}& =XF=ZX4b9J6q>#Kk|anM42 tzYQl9q_:Ed$atvEAJJa U^IGi@0 @Unknown G*Ax Times New Roman5Symbol3. *Cx Arial]Times New Roman BoldTimes7@Cambria;. *Cx Helvetica7.@Calibri?= * Courier New5. .[`)Tahoma;WingdingsA$BCambria Math"1h849849:8iZAiZA!x0 3qHP  $P]2&! xxrkruegerMAC1                           ! " # $ % & ' ( ) * + , - . / 0 Oh+'0,x   $ rkruegerNormalMAC2Microsoft Office Word@@Ln@X@Xi՜.+,D՜.+,@ hp  University of MinnesotaAZ  Titlex 8@ _PID_HLINKSA0o.http://www.itracks.com/6]mailto:casey016@umn.edu?mailto:rkrueger@umn.edu  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~      !"#$%&'()*+,-./02345678:;<=>?@EFIRoot Entry FWHData 1TableWordDocument =SummaryInformation(1DocumentSummaryInformation89MsoDataStore@R@AOAI0TOAQ==2@R@Item  PropertiesUCompObj y   F'Microsoft Office Word 97-2003 Document MSWordDocWord.Document.89q