Compliance officers are always keen to be more data-driven in the development of their programs. That means they need to identify the best compliance metrics to assess the performance of your program, and then track those metrics diligently.
So, what are the best metrics? Which ones provide the most insight to you, the compliance officer responsible for a program’s performance, and to senior executives as well, who want to know how much the compliance program supports their broader efforts to navigate the business through all its risks?
What are compliance metrics?
Compliance metrics are measurements that indicate how well your compliance program is operating. They’re important because the annals of corporate misconduct are full of scandals where the company in question had something that looked like a compliance program—but didn’t actually work. So when disaster struck, the company paid a boatload in monetary fines and remediation costs.
Compliance metrics show how well your program is or isn’t working. Without those metrics, you have no idea whether the program truly is reducing the risk of misconduct to acceptable levels; or whether the program is just a collection of hotlines, training courses, policy manuals, and related elements that nobody uses. You have no idea whether you need more staff or better technology. In short, you have no idea whether your program is effective.
Considering the emphasis that the U.S. Justice Department and so many other regulators now place on “effectiveness,” you can’t afford to leave your compliance program in that uninformed limbo. Metrics bring performance to light.
How is compliance measured?
Compliance is measured by identifying a specific point of data related to a compliance issue and then logging it into a centralized database. That’s it, really.
That answer isn’t intended to oversimplify or make light of the task. Measuring compliance can be a complicated, tedious task, where small errors magnify into grave miscalculations over time. That means you need to thoughtfully design your compliance processes so that they generate data you can collect; and astute use of technology so the data is collected and reported—those parts of your compliance program become crucial. They provide the means for you to measure your compliance performance.
The actual pieces of data you should record will vary: dates for investigations opened and closed, dollar amounts spent on investigations, issues identified by subject type, names or roles of people under investigation, types of disciplinary action doled out, and much more. You’ll need to know what fields of data should be recorded, and you’ll need technology to record those items—ideally in an automated fashion since recording data manually is a great way to make mistakes that will haunt you later.
It’s also important that your compliance technology provide robust reporting, to summon all that data and display the results in metrics that the human mind can interpret. Compliance reporting is worth a detailed analysis unto itself; suffice to say here that the more versatile your reporting capability, the better.
So what metrics should your compliance program track? Here are five that will give you a good sense of program effectiveness.
1. Mean Time to Issue Discovery
This metric gives you a sense of how quickly your program discovers a compliance issue. You calculate it by adding up all the total “issue discovery times” and dividing that number into the total number of incidents. The result is your mean time to issue discovery.
One challenge, of course, is that to calculate this number, you first need to know all those issue discovery times: the gap between when an incident started and when the compliance team discovered it. So part of your investigation process should always be to ascertain, as best as possible: When did this issue first start? You might answer that question through interviews (say, in a harassment complaint) or by data forensics (for example, an investigation into payments to risky third parties). Regardless, answering that question should always be a priority. The rest is just math.
Mean time to issue discovery can illuminate questions such as whether you have a strong speak-up culture, or the right data monitoring capabilities to find incidents as they happen. Ideally, you want to see this metric fall over time. (That is, you’re becoming aware of issues more quickly.)
2. Mean Time to Issue Resolution
This metric is a companion to the one above; it helps you understand how quickly you resolve an issue once the problem is discovered. You can also calculate this metric in roughly the same way: add up the total time for all issues to be resolved, and then divide that number by the total number of issues. The result is mean time to issue resolution.
With this metric, always beware of combining too many issues into one number, since that might blur away important information about specific types of issues. For example, if you have an excellent IT forensics team but an understaffed legal team, you might have a stellar mean time to resolution for data-intensive issues, but a terrible number for resolution of interview-heavy issues like antitrust misconduct. Consolidating both of them into a single number for all issues would make no sense. So as much as possible, track mean time for issue resolution by each type of issue, so you don’t lose that insight.
Time to issue resolution can suggest problems with resources, or problems with technology or workflows (too many manual processes, where automation could help).
3. Compliance Expense Per Issue
You can calculate this metric by dividing your total compliance budget into the number of issues your program manages, perhaps calculating it every quarter or every year.
In one sense, calculating all your issues together can make sense here: it can help to demonstrate to senior executives or the board just how much money an effective compliance program consumes.
This metric can also help to understand why certain issues cost more to resolve than others, which in turn might help you understand what solutions would make the most sense. If you spend a fortune on due diligence, you might want to introduce automation. If you spend a lot on investigations of workplace harassment or antitrust behavior, a better solution might be more training to prevent that misconduct from happening in the first place. (Or a few disciplinary measures to, ahem, demonstrate the importance of good conduct.)
4. Severity Gap Between Predicted and Actual Risks
“Negative risk events,” as we politely call them, are inevitable. Boards and the C-suite understand that which is why they spend so much time modeling risks and setting up contingency plans—including contingency accounts or insurance policies, to cover the costs of those negative events. To that end, then, you should also measure the gap between the predicted severity of a risk and its actual severity.
Ideally, you should measure that gap both financially (“This investigation cost us way more than we thought it might”) and operationally (“Those weak access controls let attackers steal twice as much data in half the time”). This helps you understand your risk assessment capabilities. Hopefully, the gap will be small, which means your assessments are sharp—but even if the gap is wide, knowing that as soon as possible is far better than not knowing it at all.
5. Risk Mitigation Timeframe
This is the time that elapses between your discovery of a risk and when you implement any changes necessary to mitigate that risk. It’s relatively easy to calculate because you’ll know the discovery date of every risk and the date when you complete any mitigation. So just add together the mitigation times for all risks, divide that number by total number of risks you monitor, and that’s the metric.
Risk mitigation timeframe is a good metric to know because it shows how well the compliance program can implement changes. Time and again we’ve seen regulators stress the importance of adapting your compliance program to changing risk circumstances; this metric lets you understand how adroitly you can make such change. With that understanding, you can then approach senior executives or the board for more or different types of resources. Or, you might decide to redesign some problematic workflows or business processes so remediating control weaknesses won’t take so much time.
The Importance of Compliance Metrics
Those are only a few possible compliance metrics, of course. We could also talk about metrics for specific subjects, such as retaliation or corruption; or metrics for tasks performed, such as due diligence and root cause analysis. We’ve barely scratched the surface here.
Still, these five metrics paint a picture of how well your program performs basic functions of issue identification and resolution. You won’t succeed over the long term without knowing them.
Matt Kelly is an independent compliance consultant and the founder of Radical Compliance, which offers consulting and commentary on corporate compliance, audit, governance, and risk management. Radical Compliance also hosts Matt’s personal blog, where he discusses compliance and governance issues, and the Compliance Jobs Report, covering industry moves and news. Kelly was formerly the editor of Compliance Week. from 2006 to 2015. He was recognized as a "Rising Star of Corporate Governance" by the Millstein Center in 2008 and was listed among Ethisphere’s "Most Influential in Business Ethics" in 2011 (no. 91) and 2013 (no. 77). He resides in Boston, Mass.