Evidence-Grounded LLM-Assisted Android Mobile Forensic Reporting

Overview & Synopsis

Large language models can assist forensic report drafting, but fluent output does not ensure evidentiary reliability. This study presents an evidence-grounded framework for evaluating LLM-assisted Android forensic reports at the atomic-claim level. Gemini, Llama, and Qwen each generated reports for two controlled cases: a public Android 13 dataset and a physical LG device with a zero-reportable-fact boundary. The six accepted reports were decomposed into 612 claims and evaluated against frozen, human-reviewed authorities through content labeling, citation validation, fact-recall analysis, independent rechecking, and dispute resolution. The final corpus contained 424 supported, 163 unsupported, four contradicted, four overstated or incomplete, and 17 excluded claims. In the public case, all three models recovered all 27 authorized report-level facts. Citation validity nevertheless differed substantially: Gemini produced 168 valid citations from 189 required citations, Qwen produced 145 from 189, and Llama produced none from 135. In the physical device case, the models generated 65 claims despite the zero reportable-fact boundary, including 38 unsupported and one contradicted claim. These results show that evidence restriction and complete fact recall do not guarantee claim support or valid evidence attribution. The framework provides an auditable method for detecting unsupported content, preserving claim-to evidence lineage, and retaining investigator authority over final forensic reports.