Intersection of Big Data and Compliance: e-Discovery

Although often misused and over-used, Big Data is far more than just a catchphrase. Insurmountable amounts of structured, semi-structured and unstructured data are bombarding corporate servers, networks, websites, phone systems, etc. each day. Although this exponential growth in data provides the foundation, it is not in and of itself Big Data. It is the ability to derive value from this data that exemplifies Big Data. As described by Arnub Gupta, CEO of Opera Solutions at Clearsight’s Big Data Executive Forum, the art of Big Data is not harnessing the petabytes of data that corporations consume each day, but extracting the signals from the noise/data that is important. Said another way, Big Data is about generating new or unique business insights from our data. So what does this mean?

From a fundamental perspective, Big Data is changing the way we are conducting business. The ability to tap into ones data assets and draw new and exciting insights, if not already, will become mission critical for virtually every business entity, academic institution, or government body. It has the promise of enabling organizations to be better informed, more efficient, effective and ultimately more profitable.

It truly is a disruptive technology. Notwithstanding its benefits, Big Data does present a whole new set of challenges and at this stage may be viewed by many corporations as more of a curse then a blessing (e.g. compliance). These viewpoints too will change as organizations begin to realize incremental value from their data assets. Big Data is already revolutionizing the way a number of organizations and industries approach or conduct business (e.g. marketing, retail, compliance, etc.). e-Discovery is no exception.

We recently attended the Palantir GovCon 8 conference held in Tysons Corner, VA (D.C.). An event by the way that is well worth the price of admission. For the uninitiated, which included me before last Tuesday, Palantir is the preeminent software platform for analyzing, integrating and visualizing data of all kinds. It was at this event that we witnessed the power of Big Data as applied to e-Discovery. Palantir has partnered with K2 Intelligence, an investigative and risk analytics consulting firm, in support of the legal team responsible for recovering the assets of Bernie Madoff’s Ponzi scheme. They are charged with, “extracting complex narratives from millions of documents, communication records and financial data from what was previously unmanageable volumes of data.”

K2 demoed an application that it built using Palantir’s open API by walking the audience through an actual workflow from the case, taking extra care to keep all identities anonymous. Without going into a play-by-play, the workflow coupled with the visualization capabilities was nothing less than extraordinary. In a matter of minutes, K2’s analyst aided by the technology platform was able to establish relationships between disparate pieces of data with different structural and temporal characteristics. As important, the visualization of the data lent itself to create a rich and intuitive narrative from which to base an argument and build a case.

In an industry that has become increasingly complex due to the deluge of electronically stored information (ESI) it now has to manage and analyze quickly – e-mail, instant messaging, social media, etc. –tools and workflows such as those exhibited by K2 and Palantir are mission critical. It’s no wonder PE and strategic investors alike are increasingly focused on providers of legal technology and services. Industry activity in this space has been very active a trend that we believe will continue if not accelerate for the foreseeable future. Below are just a few of the e-Discovery deals that have occurred in the last 18 months.