A team built a RAG system for a 500-page technical configuration manual. Users frequently ask about specific configuration settings. The system consistently returns irrelevant results even though the correct information is in the manual. The team used fixed-size 512-token chunks with no overlap.
-
Explain three specific ways the chunking strategy could be causing these retrieval failures. For each failure mode, be specific about what happens at the chunk boundary and why it breaks retrieval.
-
For each failure mode, describe what chunking approach you would use instead and why it would improve retrieval quality.
-
The manual has a hierarchical structure: Chapters → Sections → Subsections → Configuration entries. Propose a hierarchical chunking strategy that takes advantage of this structure. How would retrieval work against this hierarchical index?