Grounding: claims are tied to sources

A RAG answer without a reference to the retrieved chunk is an unverifiable claim. But a non-empty list of references guarantees nothing either: what must be checked is support for each claim individually, not whether the answer as a whole has a bibliography.

Retrieval-augmented generation was named by Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (NeurIPS 2020): condition generation on passages fetched from an index, so knowledge can be updated without retraining and answers can point at their evidence. The gap between pointing and supporting was then measured. Rashkin et al. defined attribution to identified sources (AIS) as a human-judged standard, and Liu, Zhang and Liang, “Evaluating Verifiability in Generative Search Engines” (2023), found that in the systems they audited only about half of the generated sentences were fully supported by their citations.

class Claim(BaseModel):
    text: str
    support: list[ChunkRef]
    support_status: Literal["supported", "partial", "unsupported"]

class GroundedAnswer(BaseModel):
    claims: list[Claim]                      # claims, not a wall of text

    @model_validator(mode="after")
    def must_be_supported(self):
        for c in self.claims:
            if c.support_status != "supported":
                raise ValueError(f"unsupported claim: {c.text[:60]}")
            for ref in c.support:
                if not index.exists(ref):    # the chunk actually exists
                    raise ValueError(f"phantom citation: {ref}")
                if not acl.readable(ref, self.actor):  # and this user can see it
                    raise ValueError(f"citation leaks source: {ref}")
        return self

The validator checks form. The substantive part is a separate step: an entailment check (does the chunk support the claim) and a coverage metric — what share of checkable claims has support. Both go into evals (51) because they are expensive and non-deterministic. No support → “I don’t know” is cheaper than a hallucination.

What grounding does not give. A supported claim is traceable, not true: the chunk may be stale, superseded by a later policy, or wrong. So the index needs the same freshness and provenance discipline as any data (58), and a citation should carry the document version, not just its title. Answers that combine several chunks — “A is cheaper than B” — are often only partially supported by each; treat partial as its own outcome rather than rounding it up. And the strict rule has a cost: a system that refuses everything not literally in the corpus will refuse reasonable inferences; decide per product whether “I don’t know” or a labelled inference is the better failure.