Big Data & Compliance: For Better or Worse

The explosion of Big Data, both tools for gathering and for analyzing, has created tremendous opportunities and challenges for Chief Compliance Officers. The Big Data and Compliance marriage was initially thought to be a positive transformational tool. During the honeymoon period, pundits lauded the new ability for large corporates to gather massive amounts of data in order to “ensure compliance” as well as tools to use predictive analytics to look for patterns of corruption or potential future breaches of compliance. Both services and software companies that sought to help their clients maintain compliance and report to regulators with distinct audit trails were quick to leverage the latest solutions to gather tremendous data stores to fulfill their functions.

Fast forward a couple of years and the marriage of Big Data and Compliance is fraught with both opportunities and challenges. In complex industries like Financial Services & Healthcare, tight regulatory compliance standards can be problematic. Historically, the data required to maintain regulatory compliance was housed in well-defined relational databases. In those instances where the data was unstructured, it was usually located in the email archives and tools and compliance policies were developed to manage and manipulate that data. Today, those industries are confronted with massive amounts of unstructured data. The key to ensuring compliance in dealing with Big Data is to track down and isolate the compliance-sensitive portions of that data. Because of the sheer volume, many organizations are resorting to saving everything and thus are not focusing on the critical specific elements that are important and relevant to ensuring compliance. In addition, regulators are often times requesting “all” data, which has its serious limitations to analyze appropriately and is certainly problematic for large corporates to comply with such vast requests.

To address this issue, several vendors are now offering compliance toolkits to work in a Big Data environment. We have seen IBM and EMC lead the way to help their customers address these issues. In addition, large consultancies like Deloitte, EY and PwC have been devising solutions for their clients to address these concerns both operationally and through effective use of technology. At Clearsight, we have a keen focus on both Compliance and Big Data. As a result, the intersection of those two issues and how they interact is of critical importance to us. We hear the rumblings and anticipate the use of M&A as a tool to create a more comprehensive “tool kit” to help clients respond to and comply with the many regulatory requirements that face today’s industry.