An Exploratory Sequential Deletion Audit of Small Language Models

An unsuccessful extraction prompt is weak evidence that a language model has forgotten an association. We study this distinction in a small, reproducible adaptation experiment with two sequential deletion requests. Fictional subjects receive six-digit codes; untrained control subjects use exactly the same code inventories as trained subjects, preventing marginal answer familiarity from explaining the inclusion comparison. Development runs first test whether the model learns the actual subject–code associations. Three tested LoRA recipes fail that gate, while a last-four-layer partial-dense recipe passes and is frozen before two corpus/order-seed evaluations. After deleting two groups through cheap retained-only continued adaptation, Qwen3-0.6B produces none of the 16 deleted codes under the exact greedy extraction prompt in either corpus, yet ranks the correct code first for 11/16 deleted subjects. Current-checkpoint association scores distinguish deleted from matched untrained associations with descriptive AUROC 0.8164 and 0.8477. A fixed-recipe replay on Qwen3.5-0.8B gives 1/16 and 0/16 final exact extractions, while 8/16 and 7/16 deleted associations remain top-ranked and current-checkpoint AUROC is 0.7305 and 0.6875. This exploratory replay reuses the same corpora and does not isolate a model-age effect. Exact retained association utility is preserved; paraphrase generation is not matched. Fresh retained-data references and separately labeled prior-snapshot attacks further expose the importance of attacker access. These are controlled pilot observations about an inexpensive baseline, not evidence of LoRA unlearning efficacy, a new unlearning algorithm, or a privacy guarantee.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-03
DOI
https://doi.org/10.20944/preprints202611.0220.v1
Primary Topic
Privacy-Preserving Technologies in Data
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

An Exploratory Sequential Deletion Audit of Small Language Models

Saurav Keshari Aryal
Preprints.org
Privacy-Preserving Technologies in Data
preprint

An Exploratory Sequential Deletion Audit of Small Language Models

Saurav Keshari Aryal
preprint en

Abstract

An unsuccessful extraction prompt is weak evidence that a language model has forgotten an association. We study this distinction in a small, reproducible adaptation experiment with two sequential deletion requests. Fictional subjects receive six-digit codes; untrained control subjects use exactly the same code inventories as trained subjects, preventing marginal answer familiarity from explaining the inclusion comparison. Development runs first test whether the model learns the actual subject–code associations. Three tested LoRA recipes fail that gate, while a last-four-layer partial-dense recipe passes and is frozen before two corpus/order-seed evaluations. After deleting two groups through cheap retained-only continued adaptation, Qwen3-0.6B produces none of the 16 deleted codes under the exact greedy extraction prompt in either corpus, yet ranks the correct code first for 11/16 deleted subjects. Current-checkpoint association scores distinguish deleted from matched untrained associations with descriptive AUROC 0.8164 and 0.8477. A fixed-recipe replay on Qwen3.5-0.8B gives 1/16 and 0/16 final exact extractions, while 8/16 and 7/16 deleted associations remain top-ranked and current-checkpoint AUROC is 0.7305 and 0.6875. This exploratory replay reuses the same corpora and does not isolate a model-age effect. Exact retained association utility is preserved; paraphrase generation is not matched. Fresh retained-data references and separately labeled prior-snapshot attacks further expose the importance of attacker access. These are controlled pilot observations about an inexpensive baseline, not evidence of LoRA unlearning efficacy, a new unlearning algorithm, or a privacy guarantee.

Preprints.org
Privacy-Preserving Technologies in Data
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon