AI, Law and Document Review
How AI Will Revolutionize The Legal Industry
Have you ever heard of “Document Review” in the context of legal services? It’s one of those jobs attorneys right out of law school are drafted for or given to associates as punishment. When litigation occurs or compliance deadlines need to be met, lawyers have to go through thousands, if not millions of documents and sort what needs to be provided to the other side as relevant to the matter, what may be discarded as irrelevant, what can be withheld as attorney client privileged and if there are particularly good or bad documents that help their case. It’s one of the reasons law bills are so large. Traditionally, document review in the legal field has been a time-consuming and costly labor-intensive process.
Let’s do some quick math as an example. Let’s say you have 100,000 emails and documents you have to go through. Let’s say a lawyer can go through and make all the necessary determinations at an average of 1 document per minute (60 per hour). That means it will take attorney(s) 1,667 hours to go through all 100k docs. Now, let’s say the lawyer charges $100/hr for document review services. That amounts to a bill of $166,667. You can see how this can get very expensive when we start talking about projects in the millions of documents and at review paces less than 60 docs/hr.
Artificial Intelligence and Legal Services
Artificial intelligence (AI) has the potential to revolutionize law services, particularly in areas such as document review. Machine learning has been used to great effect in the legal industry for a little over 10 years. Lawyers being the risk averse people they are have fully utilized it to the optimal degree, always requiring some level of review, sampling, validation and methodological process to ensure comprehensiveness and accuracy. Most of those statistical results have to be disclosed to the adverse party to ensure defensibility. However, this process has cut out vast quantities of documents from needing review and lowered overall review costs.
It’s about to get exponentially better.
AI-powered technologies are now about to be employed to streamline and enhance this process further, offering significant benefits to law firms and legal professionals. Talking with an E-Discovery professional incorporating AI into document review, he said “The things we are seeing with these tools is unlike anything we have ever seen before, things that we wouldn’t have believed possible a few years ago. This is going to change the game for document review.”
AI is Faster
One major impact of AI in document review is its ability to significantly speed up the process. AI algorithms can analyze and categorize documents at a much faster rate than human reviewers, allowing for rapid identification of key information. This not only saves time but also enables legal professionals to quickly assess the merits of a case and make more informed decisions. By automating repetitive tasks, AI frees up lawyers' time, allowing them to focus on higher-value activities such as case strategy, client communication, and courtroom representation. Using AI to replace permanent document reviewers is going to save time, effort and money.
AI is More Consistent and Accurate
Furthermore, AI can improve the accuracy and consistency of document review. Human reviewers are susceptible to errors, fatigue, and biases that can impact the quality of their analysis. There are also a wide variance in the quality of reviewers; some are smart, energetic lawyers right out of school making a name for themselves, while others are uninspired, permanent temp employees who can’t get hired full time. AI systems, on the other hand, can review documents with consistent precision, ensuring a higher level of accuracy and minimizing the risk of oversight. This is no small thing. Whereas technology has consistently been accurate in this regard usually to between 85-95% of the time, human reviewers collectively are much worse, and the more complicated the matter, the more erroneous they are.
AI Does Not Have to Reinvent the Wheel With Each Case
In addition, by leveraging natural language processing and machine learning techniques, AI can identify patterns, extract relevant information, and even predict outcomes based on past cases, providing valuable insights that aid legal decision-making. Imagine an employee accused of Sexual Harrassment and an AI algorithm trained to identify language and patterns consistent with harrassment. It could be done in minutes. Now imagine the company’s HR department employing the algorithm regularly on its email servers to identify such patterns before it even becomes a problem.
AI is Ultimately Cheaper
Another significant advantage of AI in document review is its potential for cost reduction. Traditional document review methods often require substantial manpower and resources, resulting in high expenses for law firms and clients. AI solutions offer a cost-effective alternative by automating much of the review process (faster, with better results), reducing the need for extensive human labor. This can lead to significant savings in terms of time, personnel, and operational costs, making legal services more accessible and affordable for clients.
Earlier and Future Models of Machine Learning in Law
Early models1 of machine learning used were more basic; advanced search features and concept clustering, putting documents to review into buckets based on similarity. As more complex modeling applications made their way into the market (logistical regression algorithms, support vector machine applications, etc)2 the quality and speed with which these programs worked improved dramatically. However, they still required a portion of the documents be categorized and reviewed by attorneys to 1) train the system effectively and 2) provide a sample to be used for validation and metrics purposes so statistical analyses can be given to the court and adverse parties on quality.
With AI now functioning as the underlying algorithm, the ability to get immediate results with little or no initial training, and quantifiable statistical analysis without an attorney reviewing a broader sample, AI machine learning will revolutionize the industry and eliminate the need to pay teams of attorneys for document review altogether.
However, it is important to note that AI should be viewed as a tool to augment human expertise, rather than replace it entirely. While AI can handle repetitive and mundane tasks, it cannot fully replace the judgment, interpretation, and contextual understanding that human lawyers bring to the table. The integration of AI technologies in document review should be done in collaboration with legal professionals who have the final say, ensuring that the technology aligns with ethical and legal standards, and that human oversight is maintained to ensure accountability and fairness.
In conclusion, AI has the potential to revolutionize document review and other aspects of law services. By automating tasks, enhancing accuracy, reducing costs, and improving efficiency, AI technologies can empower legal professionals to deliver faster, more accurate, and cost-effective services to their clients. However, the adoption of AI in the legal field should be done thoughtfully and with proper consideration of ethical and regulatory frameworks, recognizing the need for human expertise and ensuring transparency and accountability throughout the process.
PurpleAmerica’s Recommended Stories
Surprisingly, there are not a lot of books or materials written about the use of machine learning in the legal profession; at least, there aren’t a lot that do it well, as many of them are written more from the tech position and not from the legal side. So I’m going to include some websites that are good references if you want to learn more about the topic of electronic discovery.
First up, is the EDRM (Electronic Discovery Reference Model) group. This is an industry working group that discusses how data is stored and used, and how when litigation or compliance matters arise, how they manage that process. The website (edrm.net) is a great resource touching on all different aspects of the process.
Next up, is the e-discovery Team blog, written and edited by Ralph Losey. Losey is an attorney specializing in electronic discovery largely since it transitioned from paper to digital, and has followed the industry’s use of machine learning and analytics closely for the past 10-15 years. His blog covers much of the changes over that time, and includes many of his own interactions with the technology. Some of it is pretty dense, but some of the more generic entries are extremely insightful.
https://e-discoveryteam.com/
PurpleAmerica’s Obscure Fact of the Day
Testing technologies in the legal area is not as simple as it sounds. Legal issues are often more complex than just a simple search. Most clients who seek lawyers are hesitant to allow their data to be used for testing purposes. In most cases, once the matter is concluded, the data at the law firm and various vendors who processed it are destroyed or returned.
To effectively determine how well the system is doing, you need to have a publicly available dataset, with every document categorized for the different decisions for which you need to look. Vendors and Law Firms are not going to go through and pay people to categorize large datasets for this purpose. As a result, the most widely used dataset that is used was compiled by an industry group (EDRM)3 based on public Enron documents, mostly associated with Energy Deregulation with FERC and its ultimate bankruptcy and causes for it. This set is almost 20 years old and consists of about 1.6 million documents in total (although most of the time its split up into various smaller sets). Most the categories tagged within it were done so by volunteers running searches across the set and tagging particular items of interest.
There are other sets as well, but that brings up problem #2— the consistency in categorization of these sets (including the Enron Set) and quality can vary from group to group, person to person. Many of the issues sought are simplistic and narrowly construed. As a result, often testing has a level of confirmation bias to it, based on simple text used to initially categorize the documents, which doesn’t necessarily improve on simple searches.
As a result, standards in the industry are usually slow to develop, and are made more through court cases where parties contest results and a legal decision is rendered by a magistrate or special master (or an agency demands certain standards be met) or anecdotal repetitions in results people see using the technology over time that accumulate into a new industry standard.
PurpleAmerica Cultural Criticism Corner
Law is often depicted in popular culture as a luxurious and virtuous profession. They make good money. They go to court to argue grand moral themes of right and wrong. They interact with the professional elite and hobnob with major decisionmakers.
Most practicing lawyers recognize the profession is nothing like that, and far more gray than black and white. One show though did a thoroughly excellent job of portraying the profession, “Better Call Saul.” It had it’s spectrum of attorneys; the smug managing partner, the brilliant and arrogant litigator, the ambitious idealist, the boutique firms, the in-house counsel, and in Saul/Jimmy McGill, the morally ambivalent character cutting corners and drumming up business any way he can. It also demonstrated the mechanics of the law very well; how much was done outside of court, how the people with the least money are often railroaded, how the system relies on clerks, legal secretaries and smart paralegals (who often get little or no credit), how public defenders offices are overwhelmed and how prosecution often relies on just being a better negotiator.
In one episode, a promising attorney Kim Wexler (Rhea Seehorn) makes a mistake that puts the firm in a bad light. The managing partner punishes her by making her do Document Review in the basement of the firm. There are stacks of bankers boxes against the wall, only a small window for natural light, and its dark, creepy and there are only 2 other low level staff attorneys/law clerks down there doing it as well. You go through a lot of highlighters, pens and amass a pile of staples from pulling those out with that staple pulling thingy. It is the bottom rung of bottom rungs.
This is actually consistent with the time period the show takes place (early-mid 2000s). Many document reviews were still done with paper documents at the time. Paper cuts were many. Later in the 2000s, more reviews started being wholly conducted with the documents scanned in or collected digitally, which made review faster, easier and done in large computer rooms at law firms and vendors managing the reviews. It also allowed for searching across datasets so identifying important documents went faster as did review.
Since COVID, most document reviews today are done remotely.
Outstanding Tweet
Footnotes and Parting Thoughts
Let me know what you think of the page. Please share and comment!
And by “early” I mean technology circa 2005. The tech moves so fast and increases so quickly, that we are not that far removed from everything being manually reviewed on paper.
This technology came about around 2010 to the legal industry.
https://edrm.net/