The Journey to Improved Quality Assurance at Cars.

com
By Krystan Beaumont, Cars.com
At Cars.com, we are always looking at how we can make improvements to
products and processes and the Quality Assurance (QA) team is no exception
to this. In April 2013, the QA team started the journey to making big changes
to help improve reps performance and improve our survey score results. I
will be taking you through this journey and ending with where our QA
program is today.
Cars.com is a website that allows dealers and consumers to list their vehicles
they want to sell. We also have a focus on service and sell and trade for
consumers to use Cars.com during all life cycles of their cars. We have over
20,000 dealers advertising on Cars.com and over 30 million visits a month.
To support our business, my team is a part of the Operations group where we
have different teams available to our customers to help make sure vehicles
are posting correctly, that they are using our website effectively, and making
sure the products are live on the site. The QA team supports these teams by
providing feedback and support to help improve the customer experience.
Back in April of 2013, we wanted to take the customer experience to the next
level so we implemented Verint software. This software helped us to start
looking at how we were scoring the interactions with the customers and hold
the reps to a higher standard. At the time of launching Verint, we created
forms that captured all types of behaviors and expectations. We had over 20
behaviors ranging from soft skills to troubleshooting behaviors. We included
such things as using the customers name and asking additional assistance.
Once we implemented the software, we were able to create more reports to
track trends and easily identify areas of opportunity at the rep level and at
the department level. We would use these reports to coach or to create
additional trainings to support the rep.
As time went on, we continued to make changes to the form to make it more
streamlined but what we found recently was that we were not moving the
needle with QA. Our CSAT scores were good but we were still getting
feedback that issues were not getting resolved. With all this information, we
knew it was time to change up our approach on QA again.
In 2015, I attended three different conferences including the QATC
conference in Nashville. I started hearing a different approach to QA. I
realized we had a QA program from the 90s and needed to give it a new look.
The need for it was even greater with such a large percentage of Millennials

in our call center. The old QA approach wasnt really developing them
because they werent connecting with how the feedback was being
delivered. Most of their coaching was coming after the fact and it was hard
for them to connect the dots.
Because of this, we saw a need to shift the way we approached QA. From
what we were hearing, other companies were coaching more to the soft skills
and only scoring on issue resolution. We wanted to take it a step further and
find a way to help shift our QA program to a coaching model. Also, we
wanted to customize our QA program for the different types of reps. Before,
we had a one size fits all QA program, and we wanted to move to a QA
program that could help develop newer reps so they could be onboarded
faster.
We also needed to change the focus of what we scored since we didnt see
that soft skills were moving the needle to get issues resolved. Our old form
had all behaviors similarly weighted so even if they didnt resolve the issue,
they could still get a score in the 80s. We have shortened our form now from
over 20 behaviors to only five. The focus on the form is with issue resolution
with a total weight of 35 points. The other two sections on the form are
documentation which is important to our business and customer effort. This
new form now focuses on what drives customer satisfaction and sends the
message to the rep of its importance.
Before we could start to pilot this new approach, we needed to come up with
a way to track these coaching sessions to be able to hold the reps
accountable and also have a consistent message from the different coaches.
We were able to create a form that mirrored the evaluation form. This form
has the ability to show if the rep needs continued coaching on one of the
sections. The coaches also leave comments on opportunities, successes, and
next steps. Once a coach completes a form, it feeds into a dashboard our
reporting team created in Tableau. This gives everyone access to see where
the reps are before each coaching session.
We started this pilot in February 2016 and have now fully launched it to all of
the reps in our call center. This new approach has the focus on coaching
sessions and doing fewer evaluations. Before, we were QAing about 15
items per rep no matter what their tenure or performance. As for coaching
sessions, they were only getting about 1-3 coaching sessions a month. Now
we have lessened the evaluations to nine per rep. For coaching sessions,
depending on the rep, we will do two shadow sessions a week for the first
three months for a new hire and only one coaching session for a top
performer.
Since we did a pilot, we wanted to see if there was an improvement in the
new hire reps performance from the pilot group vs. new hires from our old

QA program. What we found was the new hires in the new QA program
started off with a higher average QA score in their first month. We also saw a
bigger jump in improvement during the second month. What we found was
that the reps were able to take the action items and apply them to their
interactions with the customers. Also, the form scores with the new program
could be a lot lower than our old scores, so to see higher scores from this
group in the pilot also showed they were resolving issues.
Overall, this new program is in line with what our customers want and we are
giving the reps what they need to improve. Shifting the focus on coaching to
improve instead of scoring to improve helps provide the kind of environment
that supports our reps to do better. With our coaching dashboard, this
provides a clear understanding of where the reps are at in their performance,
and we can use it to track their performance.
If anyone has questions or would like more information on our new QA
program, please feel free to reach out to me at kbeaumont@cars.com.

Are You Ready? To Be a Great Coach?

By Christel Roco, American Airlines
Are You Ready? was the overarching theme resonating throughout
American Airlines in 2015. Are you ready to restore American as the
greatest airline in the world? was the question the CEO, Doug Parker,
frequently posed to thousands of leaders in the company while they took on
the huge task of merging American Airlines and US Airways to be the biggest
airline in the world.
Being the worlds largest airline wasnt the ultimate end goal, however. The
new American Airlines aspired to be the worlds greatest airline. Senior
leadership knew this wasnt an easy task or one to be taken lightly and
developing strong leadership teams throughout the organization was the
keystone in making this happen. Managers needed to understand the critical
role they played in the organization during such an important time in the
companys history and be able to help their teams move quickly from a state
of transition to fully functioning and driving a new aligned culture. It became
evident that leadership development at the new American needed to:
increase awareness of self and others to achieve better business outcomes
and communication in an inclusive environment; engage teams and build
strong relationships and connections within those teams; and build high
performance coaching skills among leaders.
Coaching has become an essential competency for leaders at all organization
levels and American is committed to developing a strong internal coaching
culture. They recognize that one of the most important priorities for leaders
is to be a catalyst for the high performance and career acceleration of their
team members and coaching is one of the most potent tools available. For
this reason, Americans Reservations leaders partnered with Bluepoint
Leadership Development to design and deliver a powerful, experiential
workshop to help their managers excel at coaching others for high
performance.

While coaching has indeed become todays fastest growing human resource
development activity, it also is a timeless interpersonal process that great
leaders have employed for generations. The power of coaching is easy to
understand. When a person has the opportunity to discuss and reflect on
their work, unique aspirations, and most important talents with a trusted
advocate, their learning and personal commitment is almost always
dramatically increased. For most leaders, however, a significant shift in their
assumptions, relationships, and intentions was required before they could
realize their full coaching potential.
Coaching requires leaders to engage in a process of intense relationships and
challenging conversations that will inspire and enable others to shift to a new
pathway going forward. A discussion was held around a coaching impact
model that illustrates the four commonly used leadership approaches:
directing, advising, training, and coaching. While all four approaches are
frequently used by leaders, the lasting impact is only realized when the
talent owns the solution that is brought out through coaching.
During Bluepoints Leader as Coach Workshop, American Airlines
Reservations managers gained experience with a proven coaching model
that they could immediately use in their leadership roles. The key elements
that make up the Bluepoint coaching model start with earning the right to
coach and ensuring that the leaders take an inward look and then focus on
how to strengthen their reputation in the organization. The goal is to have a
leadership brand that others will gravitate to and then want to be coached
by that leader. The second part of the model addresses making solid, deeper
connections with their talent. This is critical to get to know the whole person
and therefore understand how to tailor the coaching to their specific needs,
goals, aspirations, and wants. The final piece focuses on the coaching
conversation that may include constructive criticism, acknowledgement and
affirmation, or challenging messages that need to be addressed
The workshop incorporated the very best practices of professional coaches
with Bluepoint Leaderships expertise in leadership training. Through a series
of lectures, discussions, and engaging experiences, managers were guided
through a very hands-on coaching workshop. Coaching power tools were
shared to leverage in their coaching discussions. These included how to use
silence as power, intuitional perspective, and not leaving anything unsaid
that the talent needs to hear. Finally, the art of asking effective questions
was addressed including providing the leaders with Bluepoints 60 Big
Coaching Questions that served as a valuable resource to use when
engaging in coach-like conversations. Leading as a coach often means giving
up traditional notions of management and entering the uncertain world of
personal relationships and intense conversations.

With over 6,700 employees in 22 call centers and 12 countries, Americans

Reservations department has taken Doug Parkers challenge to heart and has
made great strides in developing their leaders in order to restore the
company to greatness. Having great leaders who take great care of
employees will result in agents ultimately taking care of our customers, who
have high expectations and many airline options these days. The airline
industry is constantly changing and so are the interactions with customers.
With so many self-help options available online now, customers frequently
call airlines with more complex issues, which in turn increases the
customers chances of becoming more frustrated. Dealing with difficult
customers not only requires the right personality and skills but it also
requires the right guidance and coaching from leadership. Its important to
have a culture where relationships and conversations truly matter and are
valued and American has taken this commitment seriously.
Christel Roco presented a session on this topic at the 2015 QATC Annual
Conference. She is the Senior Manager of Quality Management at American
Airlines. Christel may be reached at christel.roco@aa.com.

Winter Survey Results

Each quarter QATC surveys the quality assurance and training community on
critical workforce topics. Approximately 70 quality assurance and training
professionals representing a wide variety of industries participated and
provided insight into this quarters survey on quality assurance around the
area of compliance with regulations.
Survey Participants
Two-thirds of the respondents in this survey work in centers with fewer than
200 agents with the largest single group having 50 to 100 agents (27.5%).
The rest are representative of a wide variety of sizes from 200 up to 500+
agents. There were no survey participants working in centers with 401 to 500
agents. The financial and utility industries drew the largest number of
participants, but other industries are also well represented.
Quality Assurance Areas of Importance

Respondents were asked which areas of quality assurance are most

important to them on their evaluation forms. Nearly half of the participants
said customer satisfaction was the most important, with script adherence
being the least important. The second most cited area of importance was
compliance with internal policies at 21% and compliance with external
regulations was third at 16%.

Compliance with Internal Standards

When asked about how many questions on compliance with internal
standards they ask on QA audit forms, approximately one-third (37%) said 1
to 3 questions and another one-third (32%) answered 4 to 6 questions, while
only 3% percent said they do not include any at all.

Compliance with External Regulations

The survey also asked how many questions on compliance with external
regulations are asked on their QA audit forms. Almost half of the participants
(48%) had 1 to 3 of these questions, while 25% indicated there are no
questions on compliance with external regulations (suggesting some
respondents had little or no external regulation). Approximately 3% use over

10 questions on this area.

Use of Speech Analytics

Sixty-three percent of the survey participants said they do not use speech
analytics to monitor calls for compliance, but almost 30% are considering
starting. Only 7% have this technology and process in place today.

Delay Between QA Evaluation and Remedial Training

When asked what the delay between QA evaluation and remedial training is
for their centers, 46% reported 1 to 5 days, and 4% said it takes more than a
month.
Ensuring PCI/HIPAA Monitoring

PCI/HIPAA compliant call recording and monitoring was not applicable to

approximately half of the respondents to the survey. Of those who do need to
ensure PCI/HIPAA recording and monitoring, approximately one-third (17%
overall) have the recording automatically interrupted during regulated
segments of a phone conversation and another 17% (8.5% overall) handle
that portion of the transaction via an automated IVR system that is not
recorded. Less than six percent do not record calls that contain regulated
information at all.

Closing Comments
For the organizations responding to the survey, meeting customer
satisfaction goals is the most important area for quality assurance focus.
However, compliance with internal and external policies and regulation are
concerns as well and both can certainly contribute to the customers
experiences. Half of the responding centers have no requirement to meet PCI
or HIPAA rules. As a result, the focus on compliance questions in the quality
monitoring process is relatively limited with a low incidence of use of

automated speech analytics tools to assist in the monitoring. It is good to see

that the delay between the identification of an issue and the remedial
training to address it is short in many centers.

Ask the Expert

QUESTION:

In our call center, one attribute we measure on our Service Quality

Evaluation Scorecard is the closing of the call. As part of that closing, we are
asking our agents to offer further assistance. Examples of this would be
Have I answered all your questions today? or Is there anything else I can
help you with? As we continue to review/update our scorecard, I wanted to
reach out to see if others have this question/attribute/skill on their
scorecard? If so, are the agents meeting this requirement and how effective
do you feel this skill is to make it a positive customer memorable
experience? Thank you for your feedback.
ANSWER:
This question was sent to the list of QATC Annual Conference attendees who
opted-in for a list after the conference. This list is a resource for attendees
after the conference. They can send questions to the list and get feedback
from other attendees. Here are some of the responses to this question:

We have that same measurement and it is a part of our closing. Its

scored with minimal points and associates achieve it about 98% of the
time. Normally our associates use Is there anything else I can help
you with? What we do run into is there are times when associates are
unable to assist the customer with an immediate resolution and it
doesnt fit into the scenario. So we would rather have our associates
just set a follow up expectation and end with a warm or sincere
closing. Some pushback is that the associates feel that we are making
them sound robotic so we have pull back on the verbiage. We allow
them to use what works for them (see examples below). As an
organization we do feel that its a part of the customer experience and
gives customers a warm feeling of the willingness to help /serve their
needs.

Examples of warm or sincere closings:

Is there anything else today?

Can I assist you with anything else today?

Any other questions you have for me?/li>

Is there anything else you are in need of today?

What else can I help you with?

Do you have any other questions?

Have I answered all your questions?

We also utilize the, Is there anything else I can help/assist/answer

during the close. Overall we are at a 95% pass rate on this specific
metric. There are occasions when it wont fit for example, an
escalated contact and then we would simply score it as an N/A. Weve
really been driving the concept of delivering a great customer
experience and working on creating a dialog with our customers vs.
hitting specific elements on the scorecard. Many of our reps have been
working the scorecard more than working the interaction and we are
trying to flip that mindset to something akin to create a great
customer experience and the scorecard will take care of itself.

We actually just updated our QA form and removed that behavior. We

found from our customer satisfaction results (CSAT) that customers
wanted their issue resolved with ease, so we took a different approach
to our QA form. We now only have three sections we focus on, which
are customer effort, issue resolution and documentation.

We recently added Offer Additional Assistance as a requirement on

our evaluation form. For the first month, we had 83% success rate for
this particular question. We felt this attribute supported our efforts for
one call resolution and organizational efficiency. We are not a scripted
call center so the agents have the flexibility to achieve this in a
manner that is most comfortable for their call flow. This attribute is not
applicable if the customer makes a statement indicating all of their
needs have been met or issues a closing, as well as if the customers
request could not be accommodated.

We have this as part of our closing and last month we had about 84%
success with that particular question on the evaluation. One of our
teams is going to a partially competency-based evaluation and while
the leadership wanted us to take that off of the procedural
requirements, the business said no, that it was important enough to
them for us to leave it. One thing to note is that it is not acceptable for
our agents to say things like Well, if theres nothing else. They
actually have to ask a question similar to your example.