How do you measure and reduce hallucination rates in enterprise RAG pipelines?

Most RAG quality issues come from bad document chunking rather than the LLM model itself. Here is how to optimize retrieval:

1. Use Semantic Structure Splitting: Split Markdown and PDFs on header boundaries (`
## `) rather than arbitrary character counts to keep tables and code blocks intact.

```python
from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
chunk_size=512,
chunk_overlap=64,
separators=["
## ", "
### ", "

", "
", " "]
)
```

2. Combine Vector + Keyword Search: Pair vector embeddings with BM25 keyword search, then pass top results to a Cohere Reranker model. This catches both semantic context and exact product/code matches.

3. Parent-Child Indexing: Store 128-token chunks for vector retrieval, but return the surrounding 1024-token parent section to the LLM.