Quality Assurance Staffing Model Action Research Project ...



Running head: QUALITY ASSURANCE STAFFING MODEL

Quality Assurance Staffing Model Action Research Project

Danielle Beck

May 16, 2009

Introduction and Problem Statement

Goal

The research goal was to collect data on how Quality Assurance analysts spend their time, analyze the data using a staffing model and present that to my director for review and evaluation.

Introduction

I am currently the supervisor of training and quality assurance for Janus Capital Group in the Fund Services department. Janus is a local mutual fund company that started as a small growth shop back in 1969. The Fund Services department is the transfer agent for the company. As the transfer agent we are responsible for taking phone calls and instructions from our direct shareholders. Direct shareholders invest in our mutual funds directly through us without going through an intermediary like a broker or dealer or supermarket such as Fidelity.

Part of my current team is the Quality Assurance team. This team’s main function is to randomly listen to phone representatives calls and listen for items that management has deemed important. As in the call center, the QA group has also gone through many incarnations. Today we consist of six analysts.

Problem Statement

The number of phone agents has decreased over the past few years, while the number of QA Analysts has remained the same. Years ago a staffing model was developed to determine if the department was right-staffed. By tracking how many calls were assessed per hour, how many hours were spent in meetings and doing project work (shrinkage), and how many hours were spent on vacation time management could determine how many units (calls assessed) were paid per hour. Management views these criteria as a critical for determining if the team is right staffed.

The problem was that the staffing model had not been updated in a couple of years and the manner of assessing phone calls was drastically changed last June to be much more subjective. By researching our current volume of work, time spent in meetings and consultations and amount of overtime being worked, a more accurate staffing model can be developed which will determine if we are right-staffed.

Background

The Fund Services department has gone through many incarnations. When I started as a phone representative 10 years ago I was a part of a department that, at the height of the technology bubble in 2000, was over 1,000 employees. Over half of those employees were phone representatives like myself. When there were over 500 phone representatives, there were dozens of QA Analysts who listened to calls all day.

Due to fluctuations in the economy, we have less than 150 employees in the department and about 50 phone representatives. However, we must still perform the job of handling phone calls from our direct shareholders and helping them with their investment decisions.

Since taking over the department in October after a company layoff I was asked by management to reevaluate the staffing model to make sure it is still relevant. The staffing model provides a method for management to determine if we have enough or too many analysts. Without an accurate picture of the volume of work and the shrinkage (meetings, reports and other assigned work) involved there is no way to determine how productive we are as a department.

In June the way we assess phone calls was changed to a much more subjective method and appears to be much more time consuming to score calls. The call assessment change was designed to align us more with our third party vendor, Prime Performance, who surveys our customers to determine their overall satisfaction with Janus and our representatives.

In these tough economic times it is crucial that we show we are right staffed. However, if there needs to be another layoff an accurate staffing model may show where cuts in QA could be made if necessary.

Research Questions

The purpose for this action research project was to create an accurate staffing model to provide management with data on how many analysts are needed to be right-staffed.

Question 1: How many calls does an analyst assess per hour?

Knowing this figure helped me determine if my assumption that it takes longer to assess a call now than it did a year ago (since we changed the way we assess calls) was valid.

Question 2: What is the average number of hours a month spent on other things such as meetings, project work and vacation time (shrinkage)?

This helped me to see where we are spending the rest of our time outside of listening to calls. Is this taking up too much of our time and where can we reduce it?

Question 3: How much overtime a month is the team working?

If the number of phone representatives has decreased and there is less work, but it takes longer to assess a call, what is a reasonable amount of overtime? How can we reduce it?

Question 4: Will the original findings change considerably after having the analysts track their time a month or two later?

I had the analysts track their time again the first week of April. This time of year we use more temporary staff (which the analysts must assess their calls as well) and because it is tax season Analysts themselves were called to the phone to help answer calls from customers (what we call reserves). This might have had a dramatic effect on the staffing model, however in the end it did not. Although, I still think it was beneficial to compare these two time frames.

My thesis is based on the data that I uncovered. I believed that based on how long it now takes to assess a phone call and with additional responsibilities and shrinkage that the Quality Assurance group’s staffing model will show that the team is right staffed.

Methods Section

My method for this project was quantitative analysis. I had a dual role as the supervisor of the group and the researcher collecting data and creating an accurate staffing model.

Participants

The participants of this action research project included the six QA Analysts, myself and my director. The analysts provided me with most of the data by filling out the Access database where they tracked their time. I analyzed the data and shared it with my director who may make decisions about our staffing levels in the future.

Data Collection Procedure

Data collection began by having each QA Analysts fill out a time tracker in an Access database. This database was created a few years ago and allows the analyst to track their time on a daily basis. The tracker requires the Analyst to enter the hours they spent on assessing phone calls, the number of calls they assessed and a variety of other items that are useful in determining shrinkage (See Appendix D).

Human Resources provided a monthly report detailing how many regular hours and overtime hours each analyst worked for the month. This helped determine where some of the overtime hours could be reduced.

|Research Question |Data Collection Method |

|1. How many calls does an analyst assess per hour? |Access database time tracker. See Appendix D for an |

| |example. |

|2. What is the number of hours a month spent on other things|Access database time tracker. See Appendix D for an |

|such as meetings, project work and vacation time |example. |

|(shrinkage)? | |

|3. How much overtime a month is the team working? |Human Resources data. See Appendix A. |

|4. Will the original findings change considerably after |Access database time tracker. See Appendix D for an |

|having the analysts track their time a month or two later? |example. |

Data Analysis

Analysts filled in the time tracker for three weeks. I sifted through the data by looking at each Analyst’s tracker and determining how many hours they spent assessing calls, how many calls they assessed and what other shrinkage they had during these three weeks. This provided me with some triangulation of the data. As Richard Sagor states in his book “Guiding School Involvement with Action Research,” “. . . it [triangulation] involves the use of multiple independent sources of data to establish the truth and accuracy of a claim.” (Sagor, 2000) Since I had all six Analysts fill out the tracker, I did not rely on just one Analyst’s time tracking, but all of them. I was then able to compare each of their time to see where there may be gaps in workload between the Analysts’s.

By reviewing and comparing each Analyst’s tracker I was able to see where we spend our time and if there are any trends. I also found out that I was right in thinking that it takes longer to review calls today than it did before we changed the way we assess calls. Based on the data the Analysts are reviewing one less call an hour than previously.

After having gone through everything, I added everything up, reviewed the vacation calendar, reviewed the meeting calendar and filled in the shrinkage numbers for January, February, and March. I took the findings to management for their approval of my methods. Management approved of the basis for my new staffing model and my shrinkage assumptions. I also had the Analysts fill out the tracker again the first week of April to validate my original findings on how long it takes to assess a call. This relates to Sagor’s definition of validity, “. . . the essential truthfulness of a piece of data.” (Sagor, 200) Going forward, I will not have the Analysts fill them out on a regular basis because I see that as micro-management. However, it will be something I will ask the Analysts to do periodically to make sure that we are continuing to stay right staffed.

The data that has been collected over the last few months will be summarized in an Excel spreadsheet. The spreadsheet will be titled QA Staffing Model 2009. An example of the spreadsheet can be seen in Appendix A - C. The spreadsheet has multiple tabs: one tab for the Shrinkage, one tab for the Bottom Line and one tab for the Units Paid.

The Shrinkage tab takes the data from the QA Analysts Access database time tracker and puts into a table. Each of the six Analysts filled out a time tracker for a total of four weeks (the last three weeks of January and the first week of April). An example of the daily time tracker is located in Appendix D. All of their time not spent assessing phone calls was categorized and added up into the Shrinkage tab. These amounts link to the Bottom Line part of the spreadsheet and the Shrinkage tab.

The Bottom Line tab takes the information from the Shrinkage tab and puts it into categories that examine how many Analysts it takes to assess back office processing work and most importantly, how many Analysts it takes to assess phone calls. This information is crucial to determine is we are right staffed. Also, this information flows into the Units Paid tab.

The Units Paid tab is designed to be a brief snapshot for management so they can view how many Analysts are required each month to do the job. Management tends to prefer small snapshots rather than viewing all the tabs that contain a lot of information to sift through.

Summary

The data that was analyzed was the Access database time trackers from each of the Analysts (see Appendix D for an example). The first time that was tracked was the last three weeks of January. Each Analyst filled out the time tracker for each day showing how they spent their time during the day. Items that were specifically looked at were hours spent assessing phone calls and how many calls were assessed and how much time was spent in meetings.

The time spent in meetings was analyzed very closely. This chunk of time goes directly to the Shrinkage tab and as well as other items such as consultations and running reports. These figures contribute to the overall number of Analysts that are needed each month to assess phone calls – which is what management is closely looking at.

To report the findings the spreadsheet will be used to show the shrinkage, the bottom line and the units paid. A table will also be used to show how the data collected answered the proposed research questions.

A report to management will be submitted that will show an accurate staffing model complete with shrinkage, number of calls assessed per hour and units paid. However, this report should also include proposals and suggestions for utilizing the QA Analysts in different ways, such as reserving in other areas and using them for special project assignments.

Schedule

1. Have analysts fill out time tracker -- 1/5/09-1/23/09

2. Review data from trackers -- 1/26/09-2/6/09

3. Wait on HR for regular and OT hours for January – 2/9/09-2/23/09

4. Simplify staffing model – 2/9/09-2/16/09

5. Fill in simplified staffing model – 2/16/09-2/19/09

6. Review findings with management – 2/23/09-2/24/09

7. Review with QA team – 2/26/09 -2/27/09

8. Have QA team fill out tracker for another week – March/early April

9. Wait on HR for regular and OT hours for February – 3/16/09-3/23/09

10. Fill in staffing model for February and March – 2/23/09-4/27/09

11. Final review with management – 4/27/09-4/30/09

Environmental Impact

The environmental impact of this research had the potential to be very positive very negative. The data showed that we are right staffed, so it helps “prove” our existence. However, if it had showed that we were over staffed, there may have been a need to realign our resources which could have affected the team negatively by lowering morale.

Also, if the findings had been negative, management may have asked some tough questions of me and my group: if there needs to be another layoff, who will be let go or how can we realign responsibilities, can someone go to part-time work? These are just some of the tough questions I still might have to face in the future. Currently, I’m not really sure how I will answer these.

Literature Review

Literature Search Methods

My literature review search methods started out promising. I spoke with the head of our scheduling department in the call center. He lent me a book about managing a call center’s phone volumes. He thought could give me an idea about staffing models, which it did. However, it did not help me understand the staffing model I was working with.

I turned to the Denver Public Library online and the University’s Library as well. I just couldn’t find anything that remotely had to do with my project. I then tried Google. I used search terms that ranged from “staffing models” to “quality assurance, call center, staffing models.” I tried different configurations of those words and unfortunately I didn’t have much success. I also tried using Google Scholar and Yahoo Answers. I had some minor success with Google Scholar, but again nothing really pertained to what I was doing for my action research. Several articles I did find were from consulting companies that provide staffing models for a fee. They did offer some free white papers on the subject that I will reference.

Literature Review Findings

The literature I did find was mostly about staffing call centers, call center workers, and one article from a completely unrelated field - transportation. There is also one article on quality assurance program, but not about staffing the QA program.

Quality Assurance Program and Cross-Training. “Getting More from Your Quality Assurance Program” (ICMI, retrieved 2009) is an article by the International Customer Management Institute and it details how a call center can get more out of their QA program. The most important idea that I took away from this article was the benefits of having an effective quality assurance program. The article describes three main benefits:

• It enables you to leverage recorded calls to identify opportunities for service improvement within the centers, as well as organizationwide improvements.

• It offers an excellent way to identify customer needs and potential long-term service issues.

• It provides another way for the call center to show the value it provides to the organization.

When I was reading this I realized that if this is how other companies use their quality assurance program, we are definitely not using our Analysts to their full potential. As a management directive, our QA Analysts listen to calls and assess them based on what management has deemed important. The QA group at Janus really doesn’t do much of anything else. The Analysts do run certain reports for management in the call assessment database, but it does appear based on this article, that there is a lot more we could be doing to add value to the call center and again help “prove” our existence.

Phil Taylor and Peter Bain describe in their paper, “An assembly line in the head: work and employee relations in the call centre,” the overall pros and cons of call center workers in the United Kingdom. Although the article is dated (1999), a lot of what the authors discuss and the mentality of call center workers, still holds true today. “’An assembly-line in the head’, always feeling under pressure and constantly aware that the completion of one task is immediately followed by another.” (Taylor & Bain, 1999) This statement is a perfect way to describe how a call center worker feels in a queue environment.

When I was reading this article I kept thinking of how I could relate it to the QA Analysts. The more I thought about it I realized that even though they do not take calls all day every day, they do listen to calls all day every day and that same mentality can be found in the Analysts. By looking for other ways that the QA group can add value to the department this may help reduce this feeling of ‘an assembly-line of the head.’

Although this article didn’t have to do with staffing models, it did give me insight into how we could add more value to the department. Another way we can add value is to cross-train the Analysts into other area.

Professors at the University of Rochester, Edieal J. Pinker and Robert A. Shumsky, looked to uncover the math behind training workers in multiple areas in their paper, “The Efficiency-Quality Trade-Off of Cross-Trained Workers.” They believe that cross-training workers can lead to economies of scale, but on the down-side it can also lead to a loss in quality. At Janus the QA Analysts not only do their job, listen to phone calls by full-time phone representatives, but they also have to help out on the phones when the phones get busy. We call this reserving. Pinker and Shumsky state, “. . . the optimal staff mix combines flexible and specialized workers.” (Pinker & Shumsky, 2000) However, the other part of the Analyst’s job is to review and assess back-office processing work and the Analysts do not currently reserve in the back-office. Recently the manager of the back-office has requested a couple of the QA Analysts who primarily deal with back-office work to reserve in the processing areas to promote cross-training efforts department wide. The question I ask myself is, “How will this impact their overall productivity?”

“In this paper we study the trade-off between the cost efficiency provided by cross-trained or generalist workers and the experience-based quality provided by specialists.” (Pinker & Shumsky) The question still remains, “How is the QA Analyst efficiency/productivity impacted when they have to stop listening to calls to actually take calls or reserve in the back office?” I am hoping to answer this the first two weeks of April. During this time, right before tax day (April 15th), the call volumes normally increase to a point that we have to have the Analysts jump on the phone to help out with queues. I plan to have the Analysts track their time again the first week of April to see the impact on calls assessed per hour when they also have to jump on the phone. This will help me with creating a more accurate staffing model.

Staffing Models. “Call Center Management on Fast Forward,” is the book on staffing models for call centers. I borrowed this book from the head of our scheduling department. Although I did not read the entire book, I focused my efforts on searching for specifics on staffing models. The biggest thing I got out of this book is that staffing levels in a call center are an art and a science. “Forecasting is the proverbial mix of art and science. It begins with predicting how many contacts you are going to get in a future period, usually a year.” (Cleveland, 2006) The book goes on how to try and predict contacts and looking at historical data.

This book gave me great insight into how our call center works. How we predict or try and forecast volumes and the appropriate staffing levels and why the scheduling department does what they do. I tried to keep my group in mind and how could use what I was reading to better interpret the data I was receiving.

However, I did find some very useful information in the white paper, “Basic Average Staffing Level Model,” by a consulting group (Tyche Consulting Group) that, for a fee, will help design and implement a staffing model. The paper describes a model that I think I can base mine on – gross staffing model. Although the gross staffing model is used to determine staffing levels for a call center, I think I might be able to adapt some of the basic concepts. “There are two basic data elements required to determine appropriate gross staffing levels: hours of direct labor required to meet the anticipated call load, and hours of direct labor available from the existing or planned staff.” (Murtagh, retrieved 2009)

Thinking about my group, I need to determine the hours of direct labor required to meet the anticipated call assessment load and the hours of direct labor available from the existing staff. I believe the tracking of how the Analysts spend their time will lead me to the direct hours of labor available to assess all the calls that management requires. Then considering the existing staff level (six analysts) I can therefore look at the shrinkage on a monthly basis which should give me my staffing model. This article was very useful to me because I was finally able to put a name, “gross staffing model,” to what I have been researching and collecting data on this whole time.

There are several different kinds of staffing models for call centers. The article “What are the various methods of determining an appropriate staffing level, again by a consulting firm, details out three different staffing models: Staffing Ratios, Gross Staffing Models and Erlang Staffing Models. Since the Murtagh article detailed out the gross staffing model, I focused my reading about the Erlang Model. “The Erlang model is based on the actual incoming traffic (calls) per 15 or 30 – minute increments. This model can be used for support organization that have multiple shifts and assists in determining how many professionals are needed for any given 15 or 30-minute time period.” (STI, retrieved 2009)

I thought their description of the Erlang model sounded familiar, so I went to go talk to my scheduling department. Not to my surprise, they explained to me that this is the model we use in our call center. Although this article didn’t help me with my research per se it did help me understand the staffing decisions that we make in our call center. By understanding the decisions on the staffing model I can then make decisions about how best to use the Analysts when they are called to the phone as reserves.

Other Staffing Models. Lastly, I found several articles from industries outside call centers. Many of them were in reference to hospital and nursing staffing. I was not able to relate them to my research. However, there was an article by the Minnesota Department of Transportation (MnDOT) that detailed out their new approach to staffing models. They spent a lot of time and money to come up with a better staffing model for all of their different workers who work on multiple projects. “The Strategic Staffing Model Plan, which is based on the premise that the future is impossible to predict with any accuracy, but that the Department can help managers plan by making reasonable assumptions about the future.” (U.S. Department of Transportation, 2002) The article goes on to describe how external and internal scans for strengths, weaknesses, opportunities and threats (SWOT) helped them develop their staffing models. Tony Cairns, a Management Analyst, states “The internal scanning process identifies how employees do something, how many employees it takes to do it, and the resources available to support it. The process helps focus on the high-priority work and the skill levels necessary to complete it.” (USDT)

This is the only article I found that had nothing to do with call centers, but that I could still see use in. The discussion around external and internal scanning and looking for SWOT areas was very thought provoking. It made me think about my department and what are our strengths, weaknesses, opportunities and threats. I feel that having the Analysts track their time can help me identify those areas. I will also look to management to see what their thoughts are on our SWOT areas.

Summary

There is a lot of information out there around call center employees and their staffing. However, as it pertains to a staffing model around a Quality Assurance team within a call center there is nothing. A complete void from what I can tell. Really, there is nothing that specifically pertains to any of the research questions presented. I was able to get some information around the value of a good Quality Assurance team and how, as the supervisor, I should be aware of the mentality and behaviors of call center representatives and my Analysts.

The various staffing models used within call centers, such as the gross staffing model and the Erlang staffing model do provide great insight into how call centers staff their phone representatives, they do not specifically address my action research. However, by adapting the gross staffing model I was able to derive some very useful information that I should be looking for in my data.

The other staffing model I reviewed from the Minnesota Department of Transportation was also very helpful in thinking about SWOT.

My action research project to come up with a viable staffing model for a Quality Assurance team seems to be a new concept. Although, I realize that staffing models in general are not new to fields outside call centers, it does seem that my research will fill this void. By looking at the Quality Assurance team as a completely separate unit from the call center to create my staffing model I see where my research may be beneficial to others.

Findings

As a review, the table below highlights my Research Questions and Analysis. It also contains a column on the findings for each research question posed earlier.

|Research Question |Analysis |Findings |

|How many calls does an analyst assess |This will help me determine if my assumption |According to the data collected on the |

|per hour? |that it takes longer to assess a call now than |Analysts weekly tracker the average number |

| |it did a year ago (since we changed the way we |of calls assessed per hour is 2. This is |

| |assess calls) is valid. |much lower than before we changed the |

| | |scoring method last June. Back then it was |

| | |over 3.5 calls assessed per hour. See |

| | |Appendix B. |

|What is the average number of hours a |This will help me see where we are spending the |As the spreadsheet in Appendix A indicates |

|month spent on other things such as |rest of our time outside of listening to calls. |Analysts spend a total of 113.5 and 231 |

|meetings, project work and vacation |Is this taking up too much of our time and where|hours per month (for the first three months|

|time (shrinkage)? |can we reduce it? |of 2009) on other items including vacation |

| | |time. This number indicates that the |

| | |Analysts use a lot of time on other items |

| | |other than assessing phone calls, which may|

| | |mean we need to look at ways of reducing |

| | |the shrinkage. |

|How much overtime a month is the team |If the number of phone representatives has |The number of overtime hours that were |

|working? |decreased and there is less work, but it takes |recorded for the first three months of 2009|

| |longer to assess a call, what is a reasonable |are: Jan – 96 hrs, Feb – 22 hrs and Mar – |

| |amount of overtime? How can we reduce it? |48. These OT hours should decrease as we no|

| | |longer have temporary phone agents. See |

| | |Appendix A. |

|Will the original findings change |I will have the analysts track their time again |After having the Analysts again track their|

|considerably after having the analysts|around March/April. This is a time of year where|time for the first week of April, the |

|track their time a month or two later?|we use more temporary staff (which the analysts |average number of calls assessed per hour |

| |must assess their calls as well) and because it |stayed the same at 2 calls assessed per |

| |is tax season analysts themselves may get called|hour. This indicates that the original data|

| |to the phone to help answer calls from customers|the Analysts provided was accurate. |

| |(what we call reserves). This could have a | |

| |dramatic effect on the staffing model and I | |

| |think it would be beneficial to compare this two| |

| |time frames. | |

Implications for Practice

The conclusions that have been drawn based on the data show that the QA team is right-staffed currently. Although, it does also indicate that if the number of phone representatives continues to decline, the team may find itself in an over-staffed position and therefore will need to find alternative ways to add value to the department.

I think that this is the biggest thing that I overlooked initially when I started this process. At the time we had 55 full time phone representatives and over 20 temporary phone representatives which meant a lot of phone calls to assess. However, over the course of the first few months of this year, the number of temporary phone representatives has dwindled and currently is at zero. We have also lost a few full-time representatives as well.

I believe the data that the Analysts have provided is accurate and valid. However, I have nothing else to compare it to and that can lead management to question the reliability and validity of the staffing model. One manager has already suggested that I have the Analysts set aside a few hours, all at the same time, where they do nothing but assess phone calls to see if my data still holds up. I believe I could do this, but I don’t know how reliable that data would be, since during the normal course of business this does not happen. The Analysts are constantly interrupted by questions from supervisors, managers, and representatives. This does not include all the meetings they attend and all the reports they run. How can having the Analysts take time to “just assess” calls be valid?

The literature review I performed shows that my research will fill a very specific gap. There is a lot of information, data and even formulas on call center staffing models, but nothing that I could find on staffing models for a quality assurance group or even a group within a call center where their workflow is based on the staffing levels of the call center.

Although I discussed possible tough questions if the findings were negative, I truly believe that given the volume of work and the amount of shrinkage, we are right staffed. However, I may be asked to lower the amount of shrinkage, possibly by trying to remove some meetings or use current meetings for multiple purposes and also to cut down on the amount of overtime in the department. And at the same time find a way to assess more calls per hour.

This action research project has added to my professional development by helping me understand how my people are spending their time and I will be able to make the case (hopefully) for the current level of staffing to management.

References

Cleveland, B. (2006). Call Center Management on Fast Forward: Succeeding in today’s dynamic customer contact environment. Annapolis: ICMI Press.

ICMI. (n.d.). Getting more from your Quality Assurance program. Retrieved March 23, 2009, from .

Murtagh, S. (n.d.). Basic average staffing level model. Tyche Consulting Group. Retrieved March 21, 2009 from

Pinker, E. & Shumsky, R. (n.d.). The efficiency-quality trade-off of cross-trained workers. Retrieved April 1, 2009 from Google Scholar:

Sagor, R. (2000). Guiding school improvement with action research. Alexandria: ASCD.

STI Knowledge Enterprise Support Solutions. (n.d.). What are the various methods of determining an appropriate staffing level? Retrieved April 1, 2009 from

Taylor, P. & Bain, P. (1999). An assembly line in the head: work and employee relations in the call centre. Industrial Relations Journal, 30(2), 101.

U.S. Department of Transportation, Federal Highway Administration. (2002). Minnesota: Innovative Practices in State DOT Workforce Management. Retrieved April 1, 2009 from .

Appendix

Appendix A – Shrinkage Tab

[pic]

Appendix B – Bottom Line Tab

[pic]

Appendix C Units Paid Tab

[pic]

Appendix D

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download