A Domain-Aware Pipeline for Structured Extraction and Retrieval-Augmented Question Answering over arXiv Superconductivity Literature

Overview & Synopsis

Scientific literature in materials physics is growing faster than researchers can manually inspect it, yet most retrieval tools rely on keyword matching or generic question answering that ignores domain-specific structure in PDF files. This paper presents an end-to-end pipeline that transforms arXiv superconductivity publications into a searchable corpus with structured metadata and retrieval-augmented question answering. The system combines thematic corpus construction, domain-aware extraction of materials and superconducting transition temperatures, and hybrid retrieval that merges dense chunk embeddings with structured publication fields. We test whether normalized domain attributes improve evidence retrieval and answer groundedness relative to lexical-only and dense text-only methods. Extraction is evaluated on a manually verified gold set of 59 superconducting materials, 46 of which have confirmed experimental critical temperatures. On the 33-material primary positive subset, domain-aware pair accuracy reaches 45.5% versus 9.1% for a regex baseline. Retrieval and generation are compared under a fixed language model (qwen2.5:7b-instruct) on a 100-question benchmark, varying only the retrieval mechanism: BM25, dense, hybrid, and hybrid with domain metadata. BM25 gives the highest source recall (Recall@5 of 47%), and adding extracted metadata to hybrid retrieval does not improve it (39% versus 41%), which we attribute to noise in the extracted fields; the main bottleneck is locating the supporting span inside a correctly retrieved paper. As a materials-informatics application, literature-derived temperatures are matched to an independent theoretical reference set: 200 materials appear in both sources, and 20 have a median experimental critical temperature above the theoretical value.