acceptodds
Under review as a conference paper at ICLR 2027

GRID: Graph Representation of Intelligence Data for Security Text Knowledge Graph Construction

Abstract

Security knowledge graphs can serve as computable and traceable external memory for security agents, yet existing LLMs lack domain knowledge grounded in real security text, and end-to-end document-to-graph training is hard to supervise with cheap and stable rewards. We present Graph Representation of Intelligence Data (G.R.I.D.), an end-to-end framework for security text knowledge graph construction. GRID builds security-domain supervision from CTI articles without manual labels, aligning each article with a traceable graph through graph extraction and knowledge-graph-conditioned text revision. It then reformulates documentto-graph learning as a scripted task bank of four-option multi-select questions and triple-level regex targets, which is cheaper and more stable than per-step LLM-judge scoring of full graphs. On a unified benchmark of 249 CTI articles from five sources (GRID, CASIE, CTINexus, MalKG, and SecureNLP), the posttrained Qwen3-4B-Instruct-2507 Task-bank Reward model reaches 84.62% sourceaveraged precision, 64.91% recall, and 68.53% Avg F1: the best recall and a near-tied top F1, with fewer tokens than the similarly strong CTINexus pipeline. The secondary End2End Reward model, trained with LLM-as-judge rewards, reaches 76.91% precision, 53.85% recall, and 58.06% Avg F1. The offline task bank, reusable across post-training runs, outperforms the online End2End reward, Choice-only Reward, and End2End SFT without RL, and both article rewriting and article-complexity ordering matter. Finally, this supervision is inexpensive to generate and reusable, in two independent senses: with the open-weight GPTOSS-120B generating the training data, the Qwen3-4B student still gains 13.8 absolute F1 points over its untrained baseline; and the task bank from our main experiments, reused with no new data generation, trains a different backbone, Llama-3.1-8B-Instruct, lifting it by 24.4 points.

Primary area:
reinforcement learning
Keywords:
knowledge graph extraction, cyber threat intelligence, reward design, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kCVUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GRID constructs CTI article-graph training pairs through traceable graph extraction and graph-conditioned article rewriting, then post-trains an LLM with an offline task bank of multi-select and regex-based rewards. It evaluates the resulting extractor on 249 CTI articles from five sources against CTI graph-construction systems and post-training variants, including rewriting, curriculum ordering, open-weight data generation, and cross-backbone task-bank reuse.

Critical strengths
  1. The paper turns a difficult open-ended extraction reward into auditable local tasks without reducing the final objective to trivial multiple-choice training.

  2. The evaluation is broader and more transparent than is typical for this application.

  3. The task-bank formulation converts open-ended KG extraction into efficiently verifiable tasks, substantially reducing reliance on expensive online LLM-as-judge rewards while enabling supervision reuse.

Critical weaknesses
  1. The Task-bank and End2End Reward methods differ substantially in training data, batch sizes, and optimization steps, with End2End receiving only 13 RL steps. Consequently, the results demonstrate cost-constrained effectiveness rather than the inherent superiority of the proposed reward formulation. A controlled comparison under matched training budgets is needed.

  2. Section 3.4 repeatedly evaluates models on the same 50-article subset and reports the best-performing checkpoints. This introduces potential selection bias, particularly since the improvement from article rewriting is only 0.005 F1. Checkpoints should be selected on a separate validation set.

  3. The proposed rewriting strategy removes security facts that are not captured by the automatically extracted KG, potentially discarding valid information due to extraction errors. This may introduce incomplete supervision and reinforce the initial extractor's omissions. The authors should evaluate how much factual information is lost during rewriting and its impact on downstream KG completeness.

Questions for authors
  1. For the RQ3 results in Table 3 and Figure 6, was the fixed 50-article evaluation subset used to choose the reported best checkpoint, or was checkpoint 100 chosen independently before inspecting this subset? If the latter, please state the pre-specified selection rule; if not, a held-out validation protocol would materially increase my confidence in the rewriting and curriculum conclusions.

  2. Were the primary Task-bank Reward checkpoint at 280 steps and the Table 1 comparison selected without repeatedly inspecting the 249-article benchmark?

Reviewer_ha6dOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GRID studies how to use a small open-source model to automatically construct knowledge graphs from Cyber Threat Intelligence (CTI) articles. The authors first use GPT-5.3 Codex to extract knowledge graphs with verbatim evidence anchors from the articles and rewrite the articles accordingly, yielding aligned article–graph data. They then reformulate document-to-graph learning as an offline-generated task bank (four-option multi-select questions and triple-level regex targets), replace the online LLM judge with scripted rewards for reinforcement learning, and order the training articles from easy to hard by article complexity. On 249 CTI articles from five sources, the post-trained Qwen3-4B model achieves source-averaged precision, recall, and F1 of 84.62%, 64.91%, and 68.53%: the highest recall, with F1 roughly on par with the strongest baseline, CTINexus (68.66%).

Critical strengths
  1. The paper proposes an offline, deterministic reward construction, so that article-to-graph RL training of small models does not have to rely on an online LLM judge. In addition, the evaluation is broad, the process is transparent, and the resources are fully open-sourced.
  2. Figure 5 reports fewer total tokens than CTINexus in the two longest article-length bins, alongside near-parity in F1. This is useful efficiency evidence, although it does not establish the complete deployment-cost comparison requested below.
Critical weaknesses
  1. GRID does not clearly outperform the strongest baseline. CTINexus merely prompts the untrained 4B model, yet achieves higher average F1 and precision; in F1, GRID outperforms CTINexus only on CASIE and on the GRID source, which the authors annotated themselves.
  2. The motivation of the paper is not sufficiently validated. The paper rests on two claims: (i) small models lack security domain knowledge, and (ii) GRID reduces deployment cost relative to commercial APIs. However, the authors never compare GRID's accuracy and cost with simply using a commercial model as the extractor.
  3. The task-bank construction is an identifiable contribution, but its incremental novelty and benefit need clearer justification. SynthIE is a relevant precedent for generating text from graph triples to train extraction; ToTTo is a table-to-text dataset, rather than an equivalent distillation/RL pipeline. GRID instead starts with real CTI articles, extracts evidence-anchored graphs, and removes uncaptured facts. Please clarify what this construction and the combined choice/regex reward contribute beyond these precedents. The use of established components alone does not establish a lack of contribution.
  4. The transfer results condition on successful outputs. In the GPT-OSS experiment, only 72 of 249 articles are valid at every checkpoint, and valid outputs fall from 247 for the untrained model to 149 for the final RL checkpoint (Appendix I). The reported 13.8-point gain therefore does not establish improvement over the full benchmark; the Llama comparison also uses a restricted intersection (the additional 21.7- and 21.8-point checks in Section 3.5 are helpful but still exclude invalid outputs). Please report full-benchmark scores, conditional quality, and validity rates, distinguish extraction failures from evaluation-service failures, and clarify failure handling in Tables 1–3. The direction of selection bias is not established.
  5. The evaluator needs validation for close system comparisons. In Appendix D.1, each calibration item receives one human assessment after the human sees the LLM verdict, allowing anchoring and providing no inter-human agreement. Please independently annotate a blinded sample from GRID and CTINexus with overlapping human labels and check the resulting relative scores. Also document how CASIE event annotations and SecureNLP token/relation annotations become reference graphs (Section 3.1; Appendix G). Text-supported precision is defensible for incomplete reference graphs; the issue is reliable application across methods.
  6. The reward ablations do not fully isolate the proposed mechanism. Section 3.3 acknowledges an unmatched End2End Reward comparison, and Appendix I reports different training configurations and only 13 online-judge RL steps. A compact comparison from the same checkpoint, with comparable data and budgets, would distinguish reward-design benefits from training differences. The current comparison supports a practical supervision-cost tradeoff more clearly than a causal advantage of the reward design.
Questions for authors
  1. Could you report the performance and per-article cost of GPT-5.4 mini, a Gemini Flash-tier model, and the teacher model GPT-5.3 Codex under the GRID prompt?
  2. How does GRID's aligned-data construction differ from existing graph-conditioned text generation methods? How does its teacher distillation pipeline differ from existing work?
  3. Can you report full-benchmark scores with validity rates and failure handling for Tables 1–3 and the RQ4 runs, a blinded human annotation with overlapping labels on a sample of GRID and CTINexus outputs, and a reward comparison from the same checkpoint with comparable data and budgets (W4–W6)?
Reviewer_QbWJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents GRID, a framework for constructing security knowledge graphs from cyber threat intelligence articles. Its main contribution is a training-data and reward pipeline: a teacher model extracts evidence-anchored graphs, rewrites the corresponding articles to remove security information not represented in those graphs, and constructs reusable multi-select questions and triple-level regex targets. A small language model is then post-trained using these scripted rewards and deployed through a two-step, ontology-guided extraction interface.

The evaluation combines 249 articles from five sources. The primary Task-bank Reward model achieves 68.53% source-averaged F1, close to CTINexus’s 68.66%, with slightly higher recall and lower token usage on longer articles. Additional experiments investigate training variants, article rewriting, curriculum ordering, open-weight supervision generation, and task-bank reuse across backbones.

The problem is practically relevant, and replacing repeated full-graph judging with reusable, locally scored supervision is appealing. However, the evidence for some central claims is weakened by evaluation conditioned on valid outputs, unmatched training configurations, and insufficient validation of the reward and data-rewriting mechanisms.

Critical strengths

Practical training formulation. The reusable task bank addresses a concrete bottleneck in RL for structured extraction: obtaining affordable rewards for open-ended outputs. Separating offline supervision generation from online reward computation is a useful design.

Attention to evidence grounding. The framework explicitly considers evidence anchors, aliases, entity hierarchy, argument roles, and unsupported deductions. These requirements make the extraction task more specific than generic entity–relation generation.

Substantial system evaluation. The study includes five evaluation sources, eight external baselines, several post-training variants, cost accounting, a human audit of the evaluator, and a separate failure analysis. The authors also acknowledge that the main reward comparison is not a matched ablation, which is important for interpreting the results.

Critical weaknesses
  1. Evaluation conditioned on valid outputs weakens the cross-backbone and open-generator conclusions.

RQ4 reports improvements on articles with valid outputs across all compared checkpoints. For the open-weight-generator experiment, Appendix I gives valid-output counts of 247, 138, 139, 161, and 149 out of 249, with only 72 articles remaining in the common-valid intersection. The reported F1 increase from 0.438 to 0.576 is computed on this intersection.

Using a common intersection makes the conditional comparison internally consistent, but it excludes failures that are themselves relevant to end-to-end extraction. In particular, the substantial decline in valid-output coverage after training could offset improvements on successfully processed articles. The current results therefore demonstrate better extraction on a selected subset, not necessarily better overall system performance.

Please define what makes an output invalid: malformed JSON, schema violations, truncation, refusal, extraction failure, or evaluator/API failure. Report performance over a fixed set of all 249 articles, with a predefined treatment of model failures and a standardized retry policy for infrastructure failures. Conditional extraction quality and valid-output coverage should be reported separately.

The additional Llama results on pairwise-valid and checkpoint-specific subsets are informative, but they still exclude invalid cases. Please also clarify whether any such exclusions affect Tables 1–3.

  1. The experiments do not sufficiently isolate the contribution of task-bank RL.

The main model receives a substantial staged training procedure: SFT, choice-question RL, continued RL with changed settings, and a final 280-step task-bank stage. By contrast, the End2End Reward model receives 13 online-judge RL steps, with different training articles, batch sizes, rollout counts, and response limits. The paper explicitly acknowledges that this is a cost-constrained system comparison rather than a matched ablation.

That comparison is useful operationally, but it cannot establish that the reward formulation itself explains the performance gap. The existing SFT and Choice-only controls also do not fully isolate the effects of training data, initialization, task mixture, and optimization budget.

The most informative additional controls would start from the same checkpoint and article pool, then compare mixed task-bank RL, regex-only RL, and SFT on the same task-bank supervision. A smaller matched online-judge experiment would help distinguish reward quality from the amount of affordable training.

Similarly, Table 1 compares complete systems, but only GRID post-trains the backbone. This is acceptable for system-level evaluation; it should not be interpreted as isolating the inference pipeline’s advantage. Running the strongest competing pipeline, CTINexus, with the post-trained backbone would provide a useful controlled comparison without requiring every baseline to be retrained.

  1. Figure quality and readability are below the standard expected for publication.

Several figures appear to be screenshot-like or low-resolution composites, with small fonts, cramped labels, inconsistent scales, and insufficient contrast.

Please redraw all figures as vector graphics (PDF/SVG), use a consistent font size and accessible color palette, make each caption self-contained, avoid screenshots of text, and check readability in grayscale and at final print size. If any figure remains unreadable in the camera-ready version, it should be treated as a mandatory correction.

Questions for authors

What prevents unsupported extra triples from obtaining the same regex reward as an otherwise correct graph?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.