How do metadata filtering and HNSW index parameters affect query latency in multi-tenant RAG systems?

Most RAG quality issues come from bad document chunking rather than the LLM model itself. Here is how to optimize retrieval: 1. **Use Semantic Structure Splitting**: Split Markdown and PDFs on header boundaries (` ## `) rather than arbitrary character counts to keep tables and code blocks intact. ```python from langchain.text_splitter import RecursiveCharacterTextSplitter splitter = RecursiveCharacterTextSplitter( chunk_size=512, chunk_overlap=64, separators=[" ## ", " ### ", " ", " ", " "] ) ``` 2. **Combine Vector + Keyword Search**: Pair vector embeddings with BM25 keyword search, then pass top results to a Cohere Reranker model. This catches both semantic context and exact product/code matches. 3. **Parent-Child Indexing**: Store 128-token chunks for vector retrieval, but return the surrounding 1024-token parent section to the LLM.