TAR can be a sticky situation when not adequately addressed in an ESI protocol

Most litigated matters involve reviewing electronically stored information that amounts to the digital equivalent of not just bankers’ boxes full of documents, but documents that, if produced in hard copy, would fill several warehouses. From a cost and efficiency perspective, lawyers must consider using technology-assisted review, or TAR, to expedite the review and controls costs. TAR is not appropriate in every scenario, but it can lower costs and expedite review in first-pass determinations of responsiveness or privilege.

Also known as predictive coding, TAR has been around for approximately 15 years. The use of TAR has been gaining wider acceptance since a magistrate judge in the Southern District of New York wrote, “computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases.” Da Silva Moore v. Publicis Groupe et al. (S.D.N.Y. 2012). In Da Silva, the magistrate judge found that when TAR is used appropriately, it reduces the costs associated with ediscovery, saves the parties time, and yields greater responsiveness when compared to manual review.

TAR has different forms. Some TAR systems utilize a “seed-set” of coded documents, while others operate based on monitoring and learning from coding in real time. A computer is trained with a seed-set of documents to result in the TAR 1.0 algorithm. So, it’s important to understand the distinction between TAR 1.0 and TAR 2.0 protocol.

TAR 1.0 is commonly used in matters where it is appropriate to produce information without an eyes-on review of all documents in the production set. The result of a TAR 1.0 workflow is usually a production of documents that were primarily reviewed by the producing party, allowing the party to comply with production obligations as efficiently as possible. As most documents are not reviewed, TAR 1.0 uses a control set to estimate performance metrics. Generally, an experienced attorney reviews the sample document set for relevance. In the typical scenario, the attorney will work with a chosen ediscovery vendor as the software “learns” how the relevant data is coded. This results in a software program to conduct a review of the remaining data with a high degree of accuracy. Studies have shown that this machine-learning to machine-reviewing predictive coding process results in a higher degree of accuracy for responsiveness than human review alone.

TAR 2.0 consists of continuous active learning, meaning the TAR systems continuously analyze the machine-learning results similar to how humans review documents. In a TAR 2.0 workflow, training of the predictive model and review of documents is performed simultaneously. Unlike TAR 1.0, which focuses on selecting the most useful training documents to efficiently build the predictive model, TAR 2.0 concentrates on reviewing all responsive documents and avoiding the review of nonresponsive documents. While TAR 1.0 relies on a control set to estimate how the model performs, TAR 2.0 generally measures performance by how well TAR finds responsive documents, though some samples are still used. This active learning protocol allows for the repeated and continued refinement of its understanding of which documents are most likely to be relevant. So, TAR 2.0 provides the reviewing attorney with a smaller and more refined bundle of documents to peruse.

A common way for TAR to find its way into a litigated matter is through an ESI protocol negotiated between the parties and ordered by the court. Responding parties face potential risks when entering into ESI protocols that are either silent or have comprehensive disclosure obligations regarding TAR.

One such risk facing parties is the situation where a court-ordered ESI protocol is in place that requires timely good faith cooperation and collaboration amongst the parties. Does it take precedence over a party’s decision to unilaterally use TAR to streamline and reduce the discovery costs associated with the review? The outcome in this situation depends on what was agreed in the negotiated protocol.

The court in Valsartan, Losartan, and Irbesartan Products Liability Litigation, 1:19-md-02875-RBK-JS (D. N.J. Dec. 2, 2020), found that the defendant violated the ESI protocol by unilaterally applying TAR without conferring with the plaintiff as agreed in the protocol. The court did not grant the relief requested by the parties. Instead, the court ordered the defendant to review its nonresponsive documents using the TAR protocol the parties negotiated but failed to consummate earlier. The plaintiff asserted that they would have negotiated keyword search terms differently had they known that the defendant intended to use TAR after applying keywords.

The court did not find this a compelling argument and refused to require the defendant to review all remaining documents. Instead, the court ordered the defendant to participate in a validation protocol that required the production of 5,000 documents identified as nonresponsive in the TAR process over the defendant’s objection. The judge also held that the dispute illustrated “the unfortunate, avoidable consequences that occur when a party does not meaningfully and timely meet, confer and collaborate regarding complex and costly ESI discovery.” Parties can mitigate this risk by considering the impact of any ESI protocol on the ediscovery process, including TAR.

Another risk facing parties relating to the ESI protocol is the situation where a party, usually the plaintiff, seeks to force the defendant to use TAR when the defendant prefers to use keyword searching, but the ESI protocol is silent as to the use of TAR. Again, the outcome here depends on what was agreed to by the parties and the actions of the parties in connection with discovery. For example, in Hyles v. New York City, 1:10-cv-03119 (Aug. 1, 2016), the plaintiffs’ application to force the use of TAR on the defendant was denied. The court cannot and will not force the defendant to use TAR against their choice of search.

Can the court compel the defendant to use a particular methodology for identifying responsive ESI? In Livingston v. City of Chicago, 16 CV 10156 (N.D. Ill., Sept. 3, 2020), the court held that the plaintiffs cannot compel the defendants to use a particular technology to identify responsive ESI. The court did permit, however, the use of TAR to identify discoverable documents and also found that culling through keyword search followed by a TAR process that integrated statistical validation “satisfies the reasonable inquiry standard and is proportional to the needs of this case under the federal rules.” In the absence of any compelling argument from the plaintiffs, the court agreed with the defendant that as the responding party, it is best situated to decide how to search for and produce emails responsive to the plaintiffs’ discovery requests. When negotiating provisions in an ESI protocol, parties must tread carefully between cooperating in good faith on the one hand and significantly surrendering the right to conduct discovery autonomously.

A peculiar risk facing parties relating to Federal Rules of Civil Procedure 26 is the situation where requesting party demands the use of TAR from an opposing party. For instance, in Lawson c. Spirit AeroSystems, 2020 WL 1813395 (D. Kan Apr. 9, 2020), the plaintiff sought to compel all documents the defendant’s TAR workflow initially deemed responsive. The defendant successfully argued that production of residual TAR documents — above an 85% recall production threshold — was disproportionate given the cost to produce the ESI and the minimal benefit (if any) of such information. The court found the second-level review and production of the residual TAR documents was not proportional to the needs of the case, citing Rule 26(b)(1). The requesting party’s insistence that an adversary use TAR despite the court’s edict that it could infer further costs met with disastrous results. Notably, the court ordered the requesting party to pay costs because of its ill-advised decision to inappropriately use TAR in the litigation process. When requesting the use of TAR in an ESI protocol, parties need to consider what is relevant and proportional from an opposing party. This is because courts can be unpredictable, more so when judicial discretion is applicable. Parties must be aware that courts can and will shift costs if discovery process becomes unduly onerous and costly for the responding party.

Valsartan and Livingston highlight particularly the risks of including TAR in a stipulated process that serves to abrogate a party’s right to determine a suitable review methodology. Additionally, parties can face challenges when they try to amend their review procedures “late in the game” or unwisely request the use of TAR in the litigation process. 