Using AI to improve a bank’s agent effectiveness

While digital is increasingly the channel that customers choose to interact with businesses, phone-based outreach is still the most commonly used engagement and sales tool. Yet, many companies lack a formulaic means of analyzing why some agents are more effective than others.

Performance analysis typically relies on supervisors listening to sampled calls, coupled with using surface-level metrics from small data sets. However, the feedback developed using this approach can be highly subjective and often generic. Being manual, the process is slow and difficult to scale, and does not provide a comprehensive view of agent performance. The inability to track metrics on an ongoing basis and across a full population of agent interactions makes it impossible to understand an agent’s performance trajectory. What’s more, it makes peer comparisons and delivering feedback difficult, which, in turn, can lead to resentment, lack of job satisfaction and, ultimately, staff churn.

To address these issues, the client needed a system that would provide greater visibility at the scale of agents’ activities (outreach, conversations, follow-up, etc.) and uncover behaviors to drive successful outcomes. The insights obtained from this analysis could, in turn, be used to develop a performance measurement and management framework with the ability to provide targeted coaching and quantify agent improvement over time.

To achieve this, the EY teams curated a robust, fact-based comparative data set. They leveraged sophisticated AI approaches to analyze conversation behaviors associated with overall progression and help identify behaviors demonstrated by top-performing agents. These insights were used to track individual team member’s performance and provide highly specific, personalized feedback. The insights were packaged into an interactive dashboard that agents and team leaders could access and use for ongoing performance review and actions.

The team needed to overcome the significant challenges of working with huge volumes of unstructured data, which is inherently noisy and often poor quality. Making the task even more complex was the difficulty of linking data from structured sources, such as customer relationship management platforms, agent hierarchy files, and performance or outcome data, to unstructured call data.

By operationalizing these insights powered by AI and machine learning, the client would be able to:

• Gain a better understanding of what top performers were doing and apply these learnings to improve training, coaching and overall performance trajectory.
• Strengthen the links between performance and incentives, and thus improve retention.
• Improve follow-up efficiency with a 360-degree view of customer outreach.
  

EY teams started by creating a “book of record” around the entirety of an agent’s interaction with a client. They captured what happened before and after across various channels – from what emails were sent and calls made, to cases opened and sales recorded.

In tandem, they used natural language processing (NLP) techniques to extract nuanced qualitative and unstructured insights from agents‘ conversations, such as topics discussed, talk track utilized and call outcome. This made it possible to gauge how agents responded to certain customer situations; whether they were invested in understanding the client’s specific situation; how they were talking about product features and functionality; and whether they were carrying out robust follow-up activity. Leveraging NLP methods overcame the data quality issues that typically drive down the performance of keyword-based approaches. These often yield unsatisfactory results due to issues including mis-transcription and contextually similar word variants.

“By addressing the quality issues inherent in conversations recorded by legacy systems and over-reliance on keywords, we were able to identify very subtle signals with a very high level of accuracy,” says Sameer Gupta, EY North America Financial Services Organization Advanced Analytics Leader.

Together with structured metrics, these foundational insights were tied to overall progression and outcome, for example, a sale recorded.

Using a variety of techniques to improve links between unstructured and structured data sets, EY teams were able to tie the foundational NLP insights to overall progression, thereby identifying gaps and opportunities by progression stage. In addition to agent-level insights, the approach helped understand and collate behaviors and methods correlated to impactful conversations and successful outcomes. The next step was to distill these insights into an interactive review dashboard. Developed with input from the client’s operations team and the end users to encourage adoption, it provides customized views for agents and team leaders that can be refreshed daily or when required.