A Controlled Study of Chunk Size Effects on Retrieval Groundedness in RAG (SQuAD v1.1)

Chunk Size Effects on Retrieval Groundedness in a RAG Pipeline (SQuAD v1.1) AbstractRetrieval-Augmented Generation (RAG) pipelines split documents into chunks before embedding and indexing, but chunking settings are often chosen heuristically. This study isolates the effect of fixed-size chunking on retrieval groundedness by holding the dataset, embedding model, and retrieval index type constant while varying only chunk size and overlap. We evaluate three word-based chunking configurations on the Stanford Question Answering Dataset (SQuAD v1.1) dev split: small (200 words, 50-word overlap), medium (400 words, 80-word overlap), and large (800 words, 120-word overlap). Chunks are embedded with Sentence-Transformers all-MiniLM-L6-v2 and indexed with FAISS using cosine similarity via an inner-product index over L2-normalized vectors. We report Answer-in-Context@k (Grounded@k) for k ∈ {1, 2, 4}, defined as whether any gold answer span appears in the concatenated top-k retrieved chunks. On a reproducible random sample of 300 QA pairs, chunking has minimal impact for k ≥ 2: Grounded@2 is 91.0% and Grounded@4 is 95.0% across all chunk sizes. At k = 1, large chunks provide a small improvement (84.7%) compared to small and medium chunking (84.3%). These results suggest chunk size matters most when retrieval is constrained to very low k, while retrieving multiple chunks reduces sensitivity to chunking on this dataset. Methods Dataset: SQuAD v1.1 dev, n = 300 sampled QA pairs (fixed seed) Chunking: Small: 200 words, overlap 50 Medium: 400 words, overlap 80 Large: 800 words, overlap 120 Embeddings: all-MiniLM-L6-v2 (Sentence-Transformers) Index: FAISS IndexFlatIP with L2-normalized vectors (cosine similarity) Metric: Grounded@k (Answer-in-Context@k), k ∈ {1, 2, 4} Key results Grounded@1: Small 84.3, Medium 84.3, Large 84.7 Grounded at 2:91.0 for all strategies Grounded at 4: 95.0 for all strategies