Tokenization is the foundational pre-processing step in every large language model (LLM) pipeline. Before an AI model reads a single word of a contract, brief, or statute, it converts the raw text into a sequence of numeric units called tokens. These tokens — which may represent whole words, word fragments (subwords), or individual characters depending on the tokenization scheme — are the actual input the model's neural network processes.
In practical terms, one token equals roughly 0.75 English words, or about 4 characters. The phrase "attorney-client privilege" tokenizes into approximately 4–5 tokens depending on the tokenizer. The phrase "indemnification" may tokenize as a single token in a vocabulary trained on legal text, but as multiple subword tokens in a general-purpose tokenizer that has not seen legal language frequently.
For legal professionals using AI tools, tokenization matters because every AI model has a hard upper limit — called a context window — measured in tokens. The context window is the maximum amount of text the model can "see" at one time. When a legal document exceeds that limit, it cannot be processed in a single pass.
Most lawyers have no need to understand the mathematics of tokenization. But the practical consequences of tokenization limits surface constantly in legal AI workflows:
Context window limits = document size limits. A 200-page M&A agreement is too long for many AI models to process in one pass. If your tool does not communicate this constraint clearly, you may receive analysis that covers only part of your document without realizing it.
Legal terminology tokenizes differently from general English. General-purpose tokenizers were trained on internet text, not Westlaw. Legal terms like "exculpatory," "subrogation," "ERISA preemption," and "cy pres" may be broken into unfamiliar subword fragments. This means the model may process them less reliably than it processes common English words — a factor in why legal AI tools with domain-specific training often outperform general-purpose models on legal tasks.
Chunking introduces analytical risk. When a document exceeds the context window, it must be divided into chunks for sequential or parallel processing. A definition of "Material Adverse Effect" that appears in Section 1 may be referenced in a closing condition in Section 8 — but if those sections fall in different chunks, the model analyzing Section 8 may not have Section 1 in its context, leading to incomplete or inaccurate analysis.
Token cost = financial cost. Most legal AI tools charge based on API usage, which is ultimately measured in tokens processed. Understanding that a 100-page document costs roughly 50,000–70,000 tokens per pass — and that some workflows make multiple passes — helps legal operations teams model AI tool costs accurately.
How It Works
Modern legal AI tools use tokenizers built on the Byte Pair Encoding (BPE) or WordPiece algorithms. These tokenizers work by:
- Starting with a base vocabulary of individual characters 2. Iteratively merging the most frequent pairs of characters or subwords into single tokens 3. Building a vocabulary of 30,000–100,000 tokens that represents the training corpus efficiently
The result is a tokenizer that represents common words as single tokens and rare or technical words as combinations of subword tokens. For a general-purpose model trained predominantly on web text, many legal terms fall into the "rare" category and tokenize into multiple subword pieces. Models fine-tuned on legal corpora — such as contracts, filings, and case law — develop tokenizers (or use the same tokenizer with better learned associations) that represent legal vocabulary more efficiently.
When you paste a contract into a legal AI tool:
- The tool's preprocessing layer receives the raw text 2. The tokenizer converts each word or subword into a token ID (a number) 3. The model receives the sequence of token IDs 4. If the sequence exceeds the context window, the orchestration layer splits the sequence into chunks 5. Each chunk is processed separately, and results are aggregated
The aggregation step — combining results across chunks — is a proprietary design decision by each legal AI vendor. Some tools use simple sequential processing; others use hierarchical summarization (summarize each chunk, then reason across summaries); others use retrieval-augmented generation (RAG) to retrieve relevant chunks on demand. Each approach has different accuracy implications.
Key Considerations for Law Firms
Know your tool's context window. Before relying on an AI tool for long-form contract analysis, verify its token limit. Tools using GPT-4 Turbo have a 128,000-token window (roughly 170 pages of dense legal text). Tools using earlier GPT-4 versions have an 8,192-token window (roughly 11 pages). This is a meaningful operational difference for M&A, real estate, or complex commercial work.
Test with your actual document types. A legal AI vendor may advertise a 100,000-token context window, but your practice area may regularly produce 300-page credit agreements. Test the tool with your real document sizes before committing to a workflow that depends on single-pass analysis.
Understand chunking disclosures. Ask your AI vendor how their tool handles documents that exceed the context window. If they use chunking, ask how cross-chunk references are handled. Reputable tools like Harvey AI and CoCounsel document their document processing architecture; if a vendor cannot explain their chunking approach, that is a red flag.
Account for token overhead. Every AI call includes system prompts, instruction text, and output space that consume tokens. A model with a 128,000-token context window does not deliver 128,000 tokens of document analysis — the system prompt, task instructions, and output tokens all consume context. Net document analysis capacity may be 60–80% of the headline context window.
Monitor token usage for cost control. Legal operations teams running AI tools at scale should instrument token usage. Running a 100-page document through an AI tool that makes three passes for extraction, risk scoring, and summary can cost 300,000+ tokens per document. At scale, this becomes a meaningful budget line item.
Limitations and Risks
Token limits create blind spots. When a document is chunked, cross-chunk references are the primary source of analytical errors. A well-structured AI tool will flag when a document exceeds its optimal processing window; a poorly designed one will silently process only a portion of the document. Lawyers must verify that the AI tool they use has actually analyzed the entire document.
Legal tokenization quality varies by training data. Tokenizers trained on general internet text produce lower-quality token representations for specialized legal vocabulary. This can lead to models that systematically underperform on legal terminology. Tools with domain-specific training — such as Luminance, which trained on a dedicated legal corpus — generally tokenize legal text more effectively.
Token count is not the same as reading comprehension. A model with a 200,000-token context window can fit a very long contract in memory, but fitting text in the context window does not guarantee accurate analysis. Context length and reasoning quality are separate dimensions. Early evidence suggests that model accuracy on long-context tasks decreases as document length increases even within the context window, a phenomenon called "lost in the middle."
Multilingual tokenization is uneven. Contracts in French, German, Spanish, or Chinese tokenize differently from English contracts, often consuming more tokens per word. Cross-border legal work using legal AI tools must account for the possibility that non-English documents consume context faster and may be processed with lower accuracy if the model's legal training data was predominantly English.