Adaptive Parsing: The LLM as the Last Line of Defense in Enterprise Document Intelligence

The quest for robust enterprise document intelligence hinges on the ability to accurately extract information from a diverse range of document formats, including complex tables, intricate diagrams, and scanned PDFs. Traditional Optical Character Recognition (OCR) methods, while adept at recovering text, often fail to preserve the structural integrity of documents. This can lead to subtle yet critical errors in information retrieval, where seemingly plausible data leads to confident, yet incorrect, answers. This article, the second in a two-part series on adaptive parsing within the "Enterprise Document Intelligence" framework, delves into the critical role of the Large Language Model (LLM) as the final safeguard against such parsing failures, building upon the foundation laid in its predecessor.

Part I of this series, "Loop engineering with adaptive PDF parsing: start cheap, pay for a heavier parser only when the page needs it," detailed the implementation of an escalation cascade and deterministic checks designed to identify and flag parsing errors at no additional computational cost. This installment focuses on the subsequent stages of the pipeline, specifically how the LLM acts as a last line of defense, and illustrates this with detailed walkthroughs of two real-world scenarios where parsing failures are detected and addressed.

The core challenge in document intelligence lies in balancing the cost and complexity of parsing with the accuracy required to answer user queries. Initial, low-cost parsing methods, such as those offered by PyMuPDF, can process a page in mere milliseconds. However, when crucial information is embedded within tables, figures, or other non-prose formats, these lightweight parsers can silently deliver incomplete or misrepresented data. Conversely, applying the most sophisticated and resource-intensive parsers to every page of every document would be prohibitively expensive and inefficient. The solution, therefore, lies in an adaptive approach: starting with the most economical parsing method and escalating to more powerful tools only when a specific need is identified within the processing pipeline.

This adaptive strategy is orchestrated through a feedback loop. The pipeline begins with a cheap parser. Its output is then subjected to a series of checks, each progressively more robust and costly. If any check signals that the initial parse is insufficient to answer the user’s question, the system escalates to a more advanced parsing method for that specific page or document section. This cascade of checks is designed to be cost-effective, with each subsequent step incurring higher computational expense but offering greater reliability. The ultimate gatekeeper in this process is the LLM itself, which, before presenting an answer to the user, reviews its own input and flags any remaining ambiguities or structural deficiencies.

The Generation-Side Checks: LLM Self-Evaluation and Groundedness

The final stages of the document intelligence pipeline involve generation and validation. Within this framework, "check 7" refers to the LLM’s self-evaluation of the parsed content it uses to formulate an answer, and "check 8" pertains to a post-generation groundedness check. While the initial parts of the pipeline are designed to catch obvious parsing failures cheaply, this article focuses on how these generation-side checks serve as the last line of defense when earlier deterministic methods have been bypassed.

It is crucial to note that the majority of user queries do not necessitate complex parsing escalations. For instance, when querying straightforward prose-based documents, the initial, low-cost parsing often suffices. However, the real test for adaptive parsing emerges when answers are concealed within structured data or visual elements.

Case A: Flat-Table Escalation, End-to-End

This scenario illustrates how a seemingly straightforward question about a table’s contents can trigger an escalation when the initial parsing method fails to capture the table’s structure accurately. The question, "What is the value of h for the base model in Table of Variations on the Transformer architecture?", targets Table 3 on page 9 of the seminal "Attention Is All You Need" paper. The correct answer, "8," is clearly present in the ‘h’ column of the ‘base’ row.

The pipeline commences with PyMuPDF, a rapid and efficient PDF parser. This initial pass yields a list of lines, each with its bounding box coordinates. While PyMuPDF successfully extracts the text for each cell in Table 3, it flattens the table’s structure, presenting each cell as an independent line of text. This format, while containing the correct data, lacks the explicit row and column relationships that a more sophisticated parser would provide.

The retrieval mechanism, employing keyword matching, successfully identifies page 9 based on the table’s caption, "Table 3: Variations on the Transformer architecture." This retrieved page, represented by the PyMuPDF-parsed lines, is then fed to the generation LLM (gpt-4.1).

In the first generation pass, the LLM correctly identifies the answer "8" and its justification points to the specific line corresponding to the ‘h’ cell. However, critically, the context_structured flag is set to False. This flag serves as an internal signal from the LLM, indicating that while it arrived at the correct answer, it did so by inferring structure from a potentially fragile, flattened representation. The LLM is essentially flagging a structural risk, even in the absence of an outright parsing error. This False value acts as the trigger for the pipeline to escalate.

Upon detecting context_structured=False, the orchestrator initiates a re-parsing of page 9 using a more robust, structure-aware parser: Azure AI Document Intelligence (Azure DI). Azure DI is designed to recognize and parse tables, converting them into a more structured format, such as Markdown. This second parsing pass yields a line_df where each row represents a complete row of the table, delimited by pipe characters, preserving the intended structure.

The second generation pass, now using the structurally sound data from Azure DI, again produces the answer "8." This time, however, the context_structured flag is True, indicating that the LLM confidently believes the underlying data structure is reliable. The cost of this escalation was minimal—an additional Azure DI call incurring a few cents and a few seconds of processing time—but the gain is significant: the answer is now structurally validated.

The annotation brick then visualizes the cited evidence by drawing a rectangle around the ‘h’ cell on the original PDF. This end-to-end audit trail allows a human reviewer to quickly verify the LLM’s answer against the source document, enhancing transparency and trust in the system.

A sidebar analysis comparing different LLM models (gpt-4o-mini, gpt-4o, gpt-4.1) across both parsers revealed that while the answer "8" remained consistent, the context_structured flag varied. More advanced models were more conservative in flagging structural risks when using the less robust PyMuPDF parser. This highlights that while upgrading the LLM can improve the detection of parsing issues, a more durable solution lies in upgrading the parser itself to eliminate the structural ambiguity.

Case B: Figure Escalation, End-to-End

This second case demonstrates the pipeline’s ability to handle queries where the answer resides within a visual element, such as a diagram. The question, "What is the architecture of the Transformer?", primarily requires an understanding of Figure 1, an encoder-decoder architecture diagram on page 3 of the "Attention Is All You Need" paper.

The initial PyMuPDF parse of page 3 captures the surrounding prose but represents Figure 1 as a placeholder type='image' row with no actual content. Consequently, the first generation pass yields a partial answer with a critical caveat: "Figure 1 is referenced in the prose but its content is absent from the parsed text." While context_structured is True (as there’s no structural parsing error in the text itself), the complete_answer_found flag is False, indicating the LLM recognizes it cannot fully answer the question.

This caveat triggers the pipeline to engage a vision-language model (a type of multimodal LLM) to process the image. The vision LLM analyzes Figure 1 and generates a structured description of the Transformer architecture, detailing components like positional encoding, multi-head self-attention, layer normalization, and residual connections. This description is then appended to the line_df as new text rows, marked with parsing_method='vision_gpt4o'.

The second generation pass, now armed with both the textual description and the structured visual interpretation, produces a comprehensive and confident answer about the Transformer’s architecture. The cost here includes the initial PyMuPDF parse, followed by a vision LLM call for the specific image, which takes longer and incurs higher costs than simple text parsing, but only for the relevant pages, not the entire document.

Stress Testing the LLM Signal: Limitations and the Importance of the Cascade

While the preceding case studies demonstrate the successful application of the LLM as a final check, a more extensive stress test was conducted to evaluate the reliability of the LLM’s self-evaluation signal in less ideal scenarios. This involved running multiple questions of increasing structural complexity against a table using different LLM models and parsers.

Across 18 runs, a significant finding emerged: when weaker LLMs produced incorrect answers due to parsing issues, they often did so with a context_structured=True flag, essentially fabricating confidence. In contrast, the only instance where context_structured=False was reliably triggered was when the LLM correctly identified the structural issue with the parse, even though the answer itself was correct. This indicates that the LLM’s binary verdict on parse quality is not consistently reliable as a primary signal for escalation.

Further experimentation with a continuous score for context_structured yielded similar disappointing results. Scores tended to clump at the high end, regardless of answer correctness, rendering the score unusable for robust decision-making.

However, when the LLM was prompted to provide a textual rationale for its assessment of the parse, it accurately described the structural deficiencies. This suggests that while the LLM’s direct verdict on structural integrity can be unstable, its ability to articulate the reasons for a problematic parse remains strong. This insight reinforces the importance of the cascaded approach, where cheaper, deterministic checks handle the bulk of error detection, and the LLM’s self-evaluation serves as a final, albeit imperfect, safety net.

Check 8: Post-Generation Groundedness

Complementing the LLM’s self-evaluation, "check 8" involves a separate LLM or Natural Language Inference (NLI) model verifying that every claim in the generated answer is directly supported by the cited source chunks. This step acts as an additional layer of validation, catching fabrications that the generating LLM might have overlooked or confidently asserted despite structural flaws. This check is critical because it employs a fresh perspective, free from the biases or confidence biases of the initial generation, and is generally more reliable and cost-effective than relying solely on the generating LLM’s self-assessment for complex issues like parsing quality.

Operational Considerations: Lazy vs. Eager Parsing and Data Model as Cache

The article also touches upon practical operational aspects. "Lazy parsing," where more expensive parsing methods are employed only when needed, is generally the preferred default for enterprise corpora, where many documents are queried infrequently. However, for documents expected to be queried repeatedly, "eager parsing"—applying the best available parser upfront—can be more efficient in the long run.

Furthermore, the proposed data model inherently provides caching. When a page is re-parsed using a more advanced method, the results are stored with the parsing_method clearly indicated. Subsequent queries referencing that page will then directly utilize the enriched, structured data, eliminating the need for redundant parsing. This mechanism allows the system to learn and optimize parsing investments based on actual usage patterns over time.

Conclusion

The adaptive parsing strategy, culminating in LLM-driven checks for self-evaluation and groundedness, represents a sophisticated approach to enterprise document intelligence. By prioritizing cost-effectiveness and reliability, this framework ensures that information can be accurately extracted from diverse document types. The cascade of checks, from inexpensive deterministic methods to the final LLM validation, creates a robust system capable of identifying and rectifying parsing errors, ultimately leading to more trustworthy and confident answers for users. The ongoing development of such intelligent systems is crucial for unlocking the full potential of unstructured and semi-structured data within enterprise environments.