When Customers and Agents Make "Noise" from Surveying
and How to Fix It
By Dr. Jodie Monger
We have learned not to take things at face value. Naturally, we seem to question everything we see,
read, observe and hear. How unnatural is it then that the management team does not question enough
about its CUSTOMER Metrics? The inherent problem lies in the types of figures we are looking at
without questioning.

Most people in contact centers anxiously await Customer Metrics because it is the validation of their
existence. Unfortunately too many assume, without questioning, that the results are reliable. Put on your
"Doubting Thomas Hat" and do not assume that it is valid or "clean" data. Without questioning, you are
likely to accept this information and make important decisions based on incorrect information. Voice of
the Customer (VOC) results are no different than any other type of research. There are biases and error
which impact the accuracy of the reporting. Do not accept results generated from "dirty" data. Dirty
data points are simply taken at face value from the customer - essentially in a raw format. This
management error puts your company, your center, your agents, and your ROI calculations that use this
data at risk.

For this education and analysis we are focusing on VOC being captured by real-time surveys. Because
let's face it, everything is moving to real-time reporting and analysis in contact centers. If you are using
a different methodology to capture customers' evaluation of contact center service; you are using
outdated programs that are no longer considered to be best practice. Metric's article "The Research
Proves It�" (available at www.metrics.net/Articles.asp) examines this issue and explains the benefits of
this methodology in contact centers.

Two Important Issues

The first area of concern is that your VOC program only collects quantitative data (rating scores or
Yes/No). By only looking at the numbers, and not giving your customers a chance to explain the reason
for them (qualitative data), you have "dirty" data. You do not know for certain that your customers
understood the questions and were using the scale correctly. The reasons behind your customers'
answers are an unknown and the concern is that you assume it to be a known. In addition, you will not
be able to explain variances in scores from one period to the next, which leads to frustration. Scores
may have changed, improved or decreased, but why?

However, asking about the WHY behind the scores does not eliminate all of the uncertainty and
concern. The second area of concern is if you are using both qualitative and quantitative methods for
your VOC data but do not apply a stringent quality control process to the data. Therefore, you are still
making assumptions and basing decisions on biased and dirty data. The errors are compounded and
magnified when used for management decision-making, agent bonuses, incentives, or other means of
recognition. Is it mostly correct? Mostly. But that doubt should raise serious flags and if used, would
place your center and agents at a serious risk.

The Cause for Concern

One of the most common errors when using both qualitative and quantitative customer data is the
assumption that all customers completed the survey correctly. Without a doubt, this is an incorrect
assumption and underscores the importance of a comprehensive quality control process BEFORE
Page 1
Copyright © 2005 Customer Relationship Metrics, L.C.Agent Customer Feedback and Staying out of Court
When Customers and Agents Make "Noise" from Surveying and How to Fix It
reporting is generated and you attempt to act on the results. Otherwise, there will be dirt in the data.
Think of this dirty data as noise - as you will be hearing about it from all levels within your
organization.

Take for example, these comments left by customers:
"I was hoping to get this resolved today. The call today went well. Chris did a good
job. It was Chris, by the way. The only thing is that I was cut off before I could get to
the customer service survey, so I had to call back."

"Please make the first 3 questions 8's. I made a mistake on grading and didn't catch it until it
was too late. I want to make sure Cheryl gets credit for a good job."

If no quality control process was in place, the comment from the first survey would have caused
the results to be assigned to another agent, Amy, who was the last agent to speak to the customer
before they completed the survey. By not only giving the customer an opportunity to leave a
comment, and then using a quality control process to clean the data, the customer scores can be
explained and the data can be moved to the proper agent, Chris. Otherwise, positive scores, as
well as negative scores will be assessed to the wrong agents - giving them and/or preventing
them from getting the proper praise or coaching that they may need.

In the second survey, which shows the most common mistake customers make, Cheryl would
have received lower scores than she deserves. You aren't as likely to hear about a good survey
being assigned incorrectly but you sure will hear about the need to increase scores when errors
are revealed. In any case, doubt about the measurement program will be embedded into the
agents' minds and give them an opportunity to dispute everything. This is noise you can not
afford. The noise from the VOC program should only be from your customer's voice and
without a quality control process in place they can not be heard correctly.

When satisfaction scores are used for incentives and bonuses, they must be fair and equitable,
which for both qualitative and quantitative means the data must be clean. Using dirty data will
lead you down the wrong avenues and will create flawed decision making processes at all levels
and create exposure to legal risks regarding human resource issues. Again, that is noise that you
can not afford to hear.

When "Close Enough" Doesn't Count

So you say that your VOC results are "close enough" and you are not concerned about this
issue. Why wouldn't you question the results to be as accurate as possible? Let us put this
"close enough" concept to the test. To what extent does the Quality Control process affect
scores for the tens of thousands of post-call surveys Metrics collects each month? On average,
between four and five percent of the surveys we collect need to be corrected. Doesn't sound
like much does it? But when you consider the use of the results and what dirty data may do to
your processes and people, it is four or five percent too much.

In order to show the actual impact of dirty data on the scores, Metrics tracked the Quality
Control process for one contact center with 110 agents and analyzed its 600 surveys. The results
are shown below. Imagine if you had used the dirty data for decision making or to reward
agents? Better yet, which numbers do you want to be held accountable to? Do you still think
close enough counts?
Page 2
Copyright © 2005 Customer Relationship Metrics, L.C.Agent Customer Feedback and Staying out of Court
When Customers and Agents Make "Noise" from Surveying and How to Fix It
In this instance, customers had much more satisfaction with their experience than initially suspected,
both for the company and the representative and a higher first contact resolution than was originally
reported. Think about the impact this has on your Contact Center ROI (Refer to
www.metrics.net/Articles.asp for more details). With dirty data, your ROI is lower and the credibility of
the results are called into question (Remember, it's natural to question and someone will question you).
Do you want to be held accountable to results that are not valid or reliable? Would you put your agents
at risk with this information? Ensure that you are indeed listening to the voice of the customer and
taking action on what they are actually saying. This will increase your Contact Center ROI, credibility
and agent satisfaction. Focus on reducing the noise so you can truly hear what the customers are telling
you.

Decreasing the Volume of Noise

As in the example above, the numbers were all higher after the quality control process than first
appeared. The other important element of this process, which is not seen on this chart, is that agent data
will be connected to the correct agent, not another agent who may have had nothing to do with the
experience except to be the last agent before the customer completed the survey. How could agents
possibly take surveys as fair and equitable if they are not reviewed and cleaned? When they get their
individual report cards, they will see the comments and know that the scores did not belong to them.
This will lead them to fight you on the entire process, which creates additional noise, both literally and
figuratively, in the process.

Noise from the front-line staff is extremely important to manage. Any metric that is part of a
performance scorecard or incentive program is intensely scrutinized. But the customers' perception of
their service needs to be a metric to which each agent is accountable. Therefore, you cannot afford to
have the important voice of the customer to be drowned out by push-backs such as:

1. The caller did not mean to give me a low score.
2. That survey score is for someone else, not me.
3. This is not fair!
Page 3
Copyright © 2005 Customer Relationship Metrics, L.C.Agent Customer Feedback and Staying out of Court
When Customers and Agents Make "Noise" from Surveying and How to Fix It
Picture here the interesting reasons why agents push-back on their Quality Monitoring scores as not
being fair and then the push-back from the Caller Satisfaction Measurement Program. All of this noise
can be deafening, but credibility issues for the VOC, and more, are addressed with a stringent, consistent
and fair Quality Control process to move the data from the "dirty" and noise amplification pile to the
"clean", believable and fair pile.

Everyone wants to be evaluated fairly. You want your agents to listen (not only hear) what the
customers are saying and to use that feedback. Energy must be spent on "how do I improve" rather than
on why "this does not apply to me". The research principles that Metrics uses along with the multiple-
step Quality Control process provides the Voice of the Customer in such a format that it can be
completely realized and leveraged within the contact center and for the organization. Silence about the
process is the quiet we need in the contact center and the noise should be coming directly and only from
the Voice of the Customer!

"Silence is wonderful to listen to."
Thomas Hardy (English poet)



Dr. Jodie Monger, is the President of Customer Relationship Metrics (www.Metrics.net),
and a pioneer in customer satisfaction research for the contact center industry. Prior to
creating Metrics she was the founding Associate Director of Purdue University's Center
for Customer-Driven Quality. Her expertise is working with organizations to help
capture and analyze the Voice of their Customer.


**For more information or to schedule an interview with Dr. Monger, please contact Jim Rembach at
336-288-8226 or Jim.Rembach@metrics.net.



Page 4
Copyright © 2005 Customer Relationship Metrics, L.C.