Performance metrics are crucial to any business operation, and that includes corporate compliance departments. Compliance officers need some way—actually, multiple ways—to measure how well their programs are working. With that said, that doesn’t mean all compliance metrics are useful.
So let’s examine three compliance metrics that industry leaders often encounter, why those metrics aren’t terribly informative—and what substitutes would be better.
Metric 1: Calls to the Hotline
“Calls to the hotline” is a pet peeve of mine and most compliance professionals. It only tells you the number of people who called the hotline, which tells you nothing useful.
More precisely, this metric says nothing about why people do or don’t call the hotline, which is what you really want to know. If you get zero calls in the quarter—is that because the corporate culture is perfect, and nobody has anything to report? Or because the culture is terrifying, and nobody dares to report anything? Both possibilities are equally true.
Even if you track the number of hotline calls over time, that still doesn’t provide much insight. Say the next quarter’s call volume goes from zero to 10. Are things 10 times worse, that finally, employees are ready to report misconduct? Or 10 times better, that employees feel empowered to speak up?
This metric doesn’t help the compliance officer understand what to do next. That’s why it’s useless.
A better solution: cross-reference the number of hotline calls against other variables. For example, measure the percentage of complaints about retaliation; or geographic region where the calls originate; or even the managerial level of the subject of the complaint.
If those percentages start to change, that’s an insight you can put to use. Perhaps the shift correlates to new policies, or new managers of a region, or simmering dissatisfaction among a certain portion of the workplace. Whatever the potential cause, now you know where to look next. That’s one hallmark of a metric that’s useful.
Metric 2: Average Case Closure Time
Average case closure time melds too many disparate issues into one blob of a metric. The time necessary to close a case depends on many details: the issue in question (allegations of accounting fraud take longer to resolve than those of workplace bullying, for example); the specific facts involved (some complaints are more complicated than others); and even the resources the compliance function has at its disposal to investigate a complaint.
Tracking average case closure time treats all those considerations equally. It ignores the nuances that exist within each business, each compliance program, and each complaint. Much like tracking calls to the hotline, this metric doesn’t help you understand why something is happening, or what you should do next.
And there’s this: when executives focus on average case closure time, the impulse will be to push that average lower. Efficiency is always welcome, but doing a rush job on cases just to achieve a lower average can lead to disaster.
A better solution: identify cases by their issue and severity, so you can track case closure times at a more granular level. For example, track average case closure times separately for accounting fraud, corruption, and workplace harassment complaints. Or track average case closure time by geographic region, and correlate that to the size of compliance staff for each region. If average case closure keeps falling in North America but is stubbornly high for Africa and the Middle East, now you know you might want to re-assign staffers across regions.
And remember: regulators don’t want fast investigations; they want competent investigations. That will always be the final analysis that matters most: the case was resolved correctly.
Metric 3: Training Completion Rate
Training completion rates tell you how many employees complete a training course. Don’t die of surprise, but some employees complete the training course and then go on to commit misconduct anyway. Gasp.
That is, training completion rates don’t give you any sense of whether employees actually absorbed the lesson, and will alter their conduct in the ways you want. But that’s what compliance officers need to know.
For example, I once had to take annual Foreign Corrupt Practices Act (FCPA) training. Every year, I skipped to the last page and took the 10-question quiz—which, thankfully, I passed every time. So clearly I knew the law, and what gifts I was or wasn’t supposed to offer according to company policy.
But does that tell the compliance officer whether I embraced the training? Not really. Plenty of employees ace their FCPA compliance training and then violate the FCPA anyway.
A better solution: consider how successful training can alter employee behavior. Then develop metrics to measure that.
For example, if you have an interactive Code of Conduct or policy manual, measure which sections of that document employees research most frequently. See if that frequency changes after certain training courses: more hits to your anti-bribery policy, or anti-harassment, or antitrust, and so forth.
Alternatively: look at calls to the compliance hotline; which subjects are rising or falling as a percentage of the total. Or compare which issues are most prevalent in your investigations function, and whether that changes in time corresponding to training courses. As always, the more granular your analysis can be, the better.
Compliance Metrics Need to Drive Insights
We could find a half-dozen additional compliance metrics that fit this theme: not helpful because they don’t help the compliance officer to understand what’s really happening in the enterprise. They’re too simple.
That doesn’t mean compliance officers should discard the concepts behind such metrics entirely. For example, you can’t ignore the number of calls to the hotline, or how long cases take to close. Those compliance metrics just need to be placed in context with other variables to generate real insight.
So when confronted with a metric that doesn’t tell you much, ask: “What do I really want to know—and what other pieces of information would go with this metric to tell me that?”
Then you’re on the path to better insight, and a better compliance program.
Matt Kelly is an independent compliance consultant and the founder of Radical Compliance, which offers consulting and commentary on corporate compliance, audit, governance, and risk management. Radical Compliance also hosts Matt’s personal blog, where he discusses compliance and governance issues, and the Compliance Jobs Report, covering industry moves and news. Kelly was formerly the editor of Compliance Week. from 2006 to 2015. He was recognized as a "Rising Star of Corporate Governance" by the Millstein Center in 2008 and was listed among Ethisphere’s "Most Influential in Business Ethics" in 2011 (no. 91) and 2013 (no. 77). He resides in Boston, Mass.