How to Measure Compliance Program Effectiveness

Having an ethics and compliance program with no implementation plan is akin to implementing the program without measuring its effectiveness. There are plenty of resources expended but no one is sure what, if anything, is gained. This post continues our discussion of ethics and compliance programs, which has covered the hallmarks of a compliant program, tone at the top, hotlines, and now we’ll address how to measure the fruits of those efforts.

DOJ Guidelines: “Evaluation of Corporate Compliance Programs”

The Fraud Section of the U.S. Department of Justice (DOJ) has indicated that its Foreign Corrupt Practices Act (FCPA) enforcement efforts will remain unchanged under the new administration. In the month after the inauguration, the DOJ’s Fraud Section issued the “Evaluation of Corporate Compliance Programs” (ECCP), a litany of “important topics and sample questions” to help companies evaluate their compliance programs. In addition, Trevor McFadden, the deputy assistant attorney general now overseeing the Fraud Section, gave a speech in which he reportedly said that FCPA enforcement and prosecution of individuals will continue to be priorities, and compliance efforts and cooperation with investigators will continue to be rewarded.

The ECCP provides a blueprint for internal reviews of compliance programs by asking questions to determine if a program is working. And if it’s not working, to determine what needs to be fixed. Hui Chen, the Fraud Section’s compliance counsel, is given credit for drafting the recent guidelines, which rely heavily on the “Filip Factors” that DOJ prosecutors use to guide their criminal investigations of corporate entities. The questions are aimed at gathering specific information about how a company implements its code of ethics or other corporate compliance program, and what steps are taken to measure its success or examine the root causes of violations.

The ECCP’s 119 questions drill down to find answers to the three basic questions we have written about before, that guide how the DOJ/SEC evaluates ethics and compliance programs, as set forth in their FCPA Resource Guide:

  • Is the company’s compliance program well designed?
  • Is it applied in good faith?
  • Does it work?

.Data Metrics: The “Hidden Gem”

The ECCP guidelines identify what companies need to ask themselves about their compliance programs, but they don’t tell companies how to go about getting the answers. As with many business operations issues, the answers are found in the data. In fact, data is called the “hidden gem” that provides a factual basis for measuring and assessing the effectiveness of ethics and compliance programs.

And the FCPA Blog says, “Data lies at the core of the [DOJ] guidance.” The author provides these examples of “compelling metrics” that reveal a program’s effectiveness:

  • How many transactions or deals were subjected to greater scrutiny because of compliance concerns?
  • Have requests for resources for compliance and control functions been denied?
  • How many internal audits have been performed in response to transactions that bore signs of bribery and corruption?
  • Where misconduct was identified, was there an investigation to find its root cause?
  • Were third parties or acquisition targets evaluated or audited for compliance issues?

This brings us to the next question: where do you find this information? Conducting employee climate surveys can help identify program strengths and weaknesses. If surveys are conducted at regular intervals they can also provide benchmarking data. For example, the data can help identify trends and determine if changes to compliance functions and controls have resulted in increased effectiveness.

In a presentation, “Ethical Culture: Defined and Measured,” the results of a company’s culture (essentially broader than climate) survey were used to compare the perceptions of its non-supervisory employees with those of its managers and executives, providing valuable insight into whether perceptions about the company’s ethics and culture are aligned throughout the company.

Data Triangulation: Test the Validity of Information

However, using one source of information may leave out important data points or allow bias to skew the data. Data triangulation involves using multiple data sources to test the validity of information. For example, other sources of information besides culture or climate surveys may include internal audit, or hotline and training data that verify or challenge the survey findings.

Under the U.S. Federal Sentencing Guidelines, one of the factors that mitigates the ultimate punishment of an organization is the existence of an effective compliance and ethics program. The DOJ/Securities and Exchange Commission (SEC) FCPA Resource Guide reinforces the need for risk-based compliance programs and an appropriate evaluation of them for continuous improvement and sustainability. For example:

  • Hotline use, response to reports, and outcomes
  • Progress of any new initiatives or compliance program enhancements
  • Training frequency and completion rates
  • Culture survey results.

Among other things, conducting culture surveys reveals how employees perceive their workplace environment and if they believe individuals at all levels of the organization are held accountable for misconduct. Additionally, surveys can measure the strength of internal controls, identify best practices, and detect new risk areas.

Research has found that culture, leadership, and values-based ethics and compliance programs increase employee reporting of misconduct and decrease retaliation. To address these issues, a list of recommended metrics includes:

  • Reviewing and updating ethics and compliance programs
  • Conducting culture surveys and knowledge assessments
  • Measuring training program reach, medium, frequency, and completion rates
  • Tracking reporting and retaliation trends by location, department, or employee
  • Identifying emerging risks through enterprise-wide risk assessments.

Make Informed Decisions Based on Data

Besides helping to create an effective compliance program, data forms the factual basis for making decisions about where resources can have the most impact. Making decisions about resource allocation based on verifiable data can move the dial from response and remediation to prevention by detecting potential problems before they happen, thereby creating a compliance program that is an effective prevention tool.

Data provides impact by measuring both the effectiveness and compliance of corporate ethics programs, and by assessing programs for outcomes and identifying problem areas such as:

  • Is the program being properly implemented?
  • Are the company’s values and ethics modeled by senior and middle management?
  • Are there sufficient control functions to detect misconduct?
  • Is there a shared commitment to ethical conduct among the company’s different components?
  • Do the company’s values and ethics play a role in making strategic and operational decisions?
  • Is there sufficient autonomy, empowerment, funding, and resources provided to the compliance function?.

As we’ve written before, “good ethics are about making good decisions, and good decisions are good for business.”

Continuously Measure Your Compliance Program 

Whether it is improving procedures to fill gaps or gathering information to perform risk assessments, data plays an important role in preventing misconduct and demonstrating a company’s commitment to effective ethics and compliance programming. Above all, data informs decision-making and provides ROI in more ways than the bottom line.

Surveying employees, conducting focus groups, analyzing existing data sources, and continuously tracking these metrics over time is a critical part of an effective ethics and compliance program.

LawRoom (powered by EverFi) delivers online training to help your business meet compliance requirements both dynamically and scalably. In addition to our award-winning online courses, LawRoom delivers a robust, cloud-based learning management system to help you easily deploy and track our growing library of ethics, anti-harassment, data security and employee conduct courses.