STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation
Abstract
STAIR leverages a table of contents to improve generative retrieval and reduce hallucinations, outperforming baseline methods on a new book-based benchmark.
Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs are improving at handling long context, but still suffer from "lost in the middle" problem. Thus, precise and accurate retrieval is important. Current retrievers chunk long context into length-based manageable chunks - in the process throwing away rich and informative semantic global structure in the corpus. We introduce a novel retrieval system STAIR that empowers an LLM to exploit global structure in a corpus such as a Table of Contents (ToC) to efficiently store and retrieve information from its model parameters. Our thorough and careful ablation studies with a finetuned Differentiable Search Index (DSI) system show that ToC helps build a low hallucination (less than 0.05%) generative Information Retrieval (IR) system and can generalize to examples where very few training samples are available. To further research in this novel direction of ToC based retrieval we release SearchTome - a diverse benchmark created from 18 books across 6 diverse domains to further research in this novel direction. STAIR achieves a high Recall@1 score of 82.6% on SearchTome as compared to DSI (76.9%), where the difference is found to be statistically significant. STAIR easily beats other strong baselines such as BM25 (59.5%), DPR (68.7%) and out-of-the-box Mistral (13.8%).
Community
Auto-STAIR: Structure-Aware Retrieval & Multi-Hop RAG Engine
We built and open-sourced Auto-STAIR, directly implementing and extending IBM Research's STAIR paper (arXiv:2609.03874) to address the future work directions in Sections 7 & 8:
- Dynamic ToC Induction (
DynamicToCInducer): Automatically synthesizes hierarchical Table of Contents trees from layout and visual cues when documents lack native outlines. - Zero-Shot Constrained Routing (
LeafTrie): Enforces prefix-tree constraints over terminal leaf nodes ($LN_D$) to achieve 0.00% structural hallucinations without needing 200 epochs of GPU fine-tuning. - Agentic Multi-Hop Reasoning (
LangGraph): Iteratively retrieves intact section text, grades sufficiency, and navigates across chapters for multi-hop queries.
- Interactive Project Space: https://huggingface.co/spaces/varun-e/Auto-STAIR
- GitHub Repository: https://github.com/Varun-Esarapu/Auto-STAIR
Get this paper in your agent:
hf papers read 2609.03874 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 3
SGK86/searchtome-subset
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper