Machine-Reads the Advisory: Benchmarking LLM Extraction of Affected-Product Ranges and Fix Information Against Curated Records

Abstract

Background: The 2024–2026 vulnerability-intelligence record created both the problem and the candidate solution for curation at scale. On the problem side: the NVD enrichment backlog (CVE submissions grew 263% between 2020 and 2025) left thousands of records without CPE, CWE, or CVSS; community responses (CISA's ADP Vulnrichment, OpenCVE Enrichment) demonstrated that automated extraction can fill the gap — OpenCVE's LLM-driven enrichment covered nearly 30% of all CVEs published in 2025 within weeks of launch. On the solution side: a benchmark literature now exists — CVE-Bench (509 CVEs, NAACL 2025), JITVul, LLM-CVX — but none measures the extraction task that curation requires. Objective: We specify the extraction benchmark that the field is implicitly running without controls: LLM extraction of affected-product ranges and fix information from advisory text, scored against curated records. Methods: Benchmark-design analysis decomposing the extraction task into six measured fields, six failure modes documented in the LLM-security literature, and six control mechanisms; alignment against the existing benchmark corpus. Results: The failure profile is field-dependent: product names extract well, version-range boundaries and exploitability claims extract poorly — exactly the fields where curation errors have the highest operational cost. Prompt injection through advisory text (E6) is the failure mode the current tooling ignores entirely, despite advisories being attacker-influenceable content. Conclusions: LLM extraction is viable as an assist to curation with confidence scoring and human review for top-severity records — and unsafe as a replacement, with version-range precision the decisive open problem.