How Large Language Models (LLMs) Are Transforming Litigation Workflows

1. Generative Pre-trained Transformers

LLMs such as GPT-4 are built upon the transformer architecture introduced by Vaswani et al. (2017). Although the transformer architecture is a concept going back decades, successful training of LLMs was enabled by recent developments in computing power to train on extremely large data sets. These models rely on a mechanism called self-attention, which enables the LLM to weigh the relationships between various elements of the input text within a context window, allowing for nuanced responses considering context, syntax, and semantics. This allows the model to perform a wide range of language-based tasks, from summarization and translation to legal analysis and argumentation. Ultimately, LLMs are next-token prediction models — they generate essentially one word at a time and are able to produce large amounts of syntactically correct responses with semantic validity.

2. Training on Massive Datasets

LLMs are pre-trained on diverse text corpora including legal cases, government filings, websites, books, and other digital documents. While these corpora are vast, they are not exhaustive, and access to proprietary or up-to-date legal databases is not included in publicly available foundation models. Consequently, general-purpose LLMs will not possess relevant domain knowledge for many tasks, which can lead to responses that contain semantic errors, often called hallucinations. This is akin to a law student who has studied every proceeding, brief, and manuscript in a law library but is asked to write a summary of a document they haven't seen.

To overcome the problem of hallucinations, critical domain information can be added to the prompt in the context window to give the model a concrete foundation for its responses. Systematic approaches to do this across specialized document databases exist such as retrieval-augmented generation; however, these approaches require nuance and tuning for the domain to operate accurately. Additionally, context window sizes are limited to hundreds of pages of text, so for domains where more specialization is necessary or style adaptation is required, fine-tuning foundation LLMs is an option.

3. The Black Box Problem

While outputs from LLMs can be highly syntactically and/or semantically accurate, the internal mechanisms of the LLM that were used to lead to the output remain largely opaque — a phenomenon referred to as the "black box" problem. The model's decisions result from complex internal representations that are difficult to trace, and while LLMs can be asked to justify their output, there are no guarantees that the justifications are accurate or representative of the underlying decisions made internally in the LLM.

4. Tokens and Context Windows

Rather than processing words, LLMs break inputs into tokens, which are typically short character sequences. Each model has a token limit, or "context window," which determines the amount of text it can process in a single session. GPT-4 Turbo, for example, supports up to 128,000 tokens, which is approximately the length of a 300-page novel. The latest LLMs can have context windows that support more than a million tokens, but LLMs may not pay attention to everything in the context window. Research has shown that LLMs tend to ignore context in the middle of the context window, preferring to pay attention to context toward the beginning and end.

What Is RAG?

RAG combines an LLM with a high-performance database of relevant legal documents. When a user submits a query, the system retrieves pertinent documents (e.g., contracts, filings, depositions) and feeds them into the LLM's context window. This approach grounds the model's responses in authoritative sources without retraining the model, enhancing both accuracy and security.

Chunking and Vectorization

Before documents can be used in RAG, they are divided ("chunked") into logical segments and converted into vector embeddings for efficient semantic search. Only the most relevant chunks are provided to the LLM, reducing irrelevant details and improving the quality of responses. The chunking strategy, embedding model, and retrieval algorithm all critically influence RAG performance and the legal usefulness of responses.

This is a differentiating feature of emerging software applications serving the legal field and is critical to both limiting hallucinations (fabricated information) and providing verifiable citations.

Model Plug-and-Play

Because RAG separates the knowledge base from the language model, newer or better LLMs can be swapped into existing pipelines without re-architecting the system. This modularity ensures legal organizations stay up to date with the latest models while maintaining their proprietary knowledge infrastructure. Eventually, companies with specialized LLM applications will submit benchmarking of their use of various foundational models to guarantee that they are getting optimum performance for the task at hand, to differentiate their product.

Common Use Cases

LLMs are reshaping litigation workflows by streamlining and enhancing a wide range of legal tasks that traditionally require substantial time and manual effort:

Effective Prompts Are:

Advanced Prompting Techniques