A RAG graph says a document was deleted, but its relations remain

A policy document is withdrawn, but an assistant still cites a relation derived from it. An open LlamaIndex issue describes one mechanism: a ChunkNode without an explicit ID derives identity from Python's process-randomized string hash. After persistence and restart, its loaded ID can differ from the key and relation endpoint saved in the graph. A delete call using the loaded ID then leaves the original node and triplet behind.

Reproduce across a restart

Create a tiny graph with one chunk and one relation, persist it, load it in a second process and delete the chunk. Query both nodes and relations afterward. Running the test in only one process can miss the identity change. Record the graph store, library version and whether IDs were supplied explicitly. A successful API response alone is not proof of deletion.

Treat removal as a data lifecycle operation

For an enterprise knowledge system, deleting a source should remove its searchable text, embeddings, graph edges, cached answers and any derived summary that can still surface its claims. Keep a source-to-artifact manifest and verify the removal at each layer. If a library defect is suspected, quarantine the affected source and avoid claiming deletion is complete until queries and graph inspection agree.

Fix identity before rebuilding trust

Prefer stable, explicit source and chunk IDs in new ingestion where supported. For existing persisted stores, plan a migration or repair that preserves the saved graph keys; simply re-ingesting can create duplicates. Test deletion on a copy of production data, then reconcile orphaned relations and stale index entries.

Adovo’s view

A knowledge system must remember what it knows and honor what it has been told to forget. Deletion is a verified outcome, not an API call. If old evidence survives, every answer inherits that governance failure.