acceptodds
Under review as a conference paper at ICLR 2027

BALANCE-DEE: Balancing Event Candidate Space Expansion and False-Positive Correction for Document-Level Event Extraction

Abstract

Document-level event extraction (DEE) systems often miss non-salient events and make cross-event argument assignment errors. Expanding the event candidate space can recover missed events, but it also introduces spurious candidates and increases structural conflicts. We study this coverage–correction tradeoff and propose BALANCE-DEE, a structured refinement framework that jointly evaluates candidate-event validity and state-conditioned action gain. A discriminatively enhanced schema guides a frozen large language model to propose potentially missed events; a two-factor eventness assessment based on event instantiation and textual support filters unsupported candidates; and a learned action-gain function scores insertion, deletion, and split-and-reallocation edits relative to the current event-record state. The model iteratively executes positive-gain edits and recomputes state-conditioned gains after each update. Across FNDEE, CrudeOilNews, and SENTiVENT, BALANCE-DEE improves trigger and argument F1 over the corresponding base extractor. On FNDEE, it raises overall event recall by 2.3 percentage points and non-salient event recall by 4.2 points, reduces over-extracted arguments by 13.2%, and lowers the cross-event argument attachment rate from 20.4% to 12.9%. These results show that high-recall candidate expansion is most effective when coupled with state-aware validation and structured correction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
event extraction, document-level event extraction, information extraction, structured prediction, action gain, nlp

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ifEtOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work focuses on the well-known coverage-correction tradeoff in document-level event extraction (DEE), where efforts to expand candidate space and recover overlooked non-salient events frequently result in spurious candidates and structural conflicts between extracted events. The authors propose BALANCE-DEE, a structured refinement framework that uses discriminatively enhanced schemas to guide candidate generation, a two-factor eventness assessment to filter invalid candidates, and a learned action-gain function to evaluate insertion, deletion, and split-and-reallocation edits. Experiments across three datasets (FNDEE, CrudeOilNews, SENTiVENT) show consistent improvements in event recall, particularly for non-salient events, while reducing over-extracted arguments and cross-event argument attachment errors.

Critical strengths

Critical Strengths

1.The paper addresses an important practical tradeoff in document-level event extraction between recall and precision.

2.The experimental results on non-salient event detection show interesting trends that could inform future work.

Critical weaknesses

Critical Weaknesses

1.Fundamental evaluation flaws: The paper's evaluation uses an overly permissive event matching criterion that artificially inflates performance scores. The reported improvements do not hold when using standard evaluation metrics for event extraction.

2.Lack of novelty: The iterative correction approach is a straightforward combination of existing candidate generation and filtering techniques that have been explored in prior work on structured prediction refinement.

3.Limited practical utility: The proposed approach increases inference time by 300% compared to single-pass extractors, making it unsuitable for most real-world applications that require low-latency processing.

4.Unconvincing gain function design: The learned action-gain function lacks theoretical justification and shows high variance across different datasets, suggesting it may not generalize well.

Questions for authors

None

Reviewer_mPvWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BALANCE-DEE addresses event under-extraction, spurious event predictions, and cross-event argument attachment errors through three parts: a schema-guidance mechanism to expand event candidates, a collaborative verification mechanism for event instantiation and textual support to filter invalid candidates, and an action-gain-guided iterative correction strategy to refine event predictions through insertion, deletion, and split-and-reallocation operations.

Critical strengths

A coherent framework for document-level event extraction. BALANCE-DEE follows a progressive pipeline: the discriminatively enhanced schema-guidance mechanism expands candidate coverage to recover missed events; the collaborative verification mechanism for event instantiation and textual support filters false-positive candidates; and the action-gain-guided iterative correction strategy then refines event records to correct argument extraction and assignment errors. Comprehensive empirical evaluation. Experiments on three datasets, together with component ablations and mechanism analyses, support the effectiveness of the proposed framework.

Critical weaknesses

Incremental methodological novelty. BALANCE-DEE integrates a discriminatively enhanced schema-guidance mechanism, a collaborative verification mechanism for event instantiation and textual support, and an action-gain-guided iterative correction strategy. While experiments support their effectiveness, the methodological contribution remains incremental. Limited cross-dataset validation of component contributions. Although BALANCE-DEE is evaluated on three datasets, component ablations and mechanism analyses are reported primarily on FNDEE. Ablations on at least one English dataset would help establish whether the contributions of the proposed components generalize across datasets.

Questions for authors

Table 5(a) reports a 4.2 percentage-point Recall improvement on the Non-salient subset. Does this gain persist when the threshold used to define the Non-salient subset is varied? A brief sensitivity analysis would help establish the robustness of this result. In addition, the authors need clarify whether the main component-level trends observed on FNDEE also hold on CrudeOilNews and SENTiVENT. The ablation experiments on these datasets will help establish the generality of the findings.

1 comment
AuthorsOpenReview ↗

Response to Reviewer mPvW

Thank you for your careful review and constructive suggestions. We have conducted additional experiments to address the points raised, and the corresponding results and analyses will be incorporated into the revised manuscript.

Q1:

We agree with the reviewer's suggestion to further examine the robustness of the recall gains for non-salient events. Accordingly, we conducted an additional sensitivity analysis of the textual-salience threshold. Keeping the model predictions fixed, we repartitioned the events using the 40th, 50th, and 60th percentiles of the training-set salience scores.

Additional Experiment 1: Sensitivity of Event Recall to Textual-Salience Thresholds on FNDEE.

Percentile40%50%60%
# Non-salient events1,7682,2182,687
Non-salient ΔR (pp)+4.9+4.2+3.8
Salient ΔR (pp)+0.6+0.4−0.1

The results show that non-salient events consistently achieve larger recall gains across all three thresholds, confirming the robustness of this trend. Since this analysis relies on a textual-salience proxy, we will further clarify its definition and limitations in Appendix B.2. The additional sensitivity analysis will be included in Appendix A.8.

Q2:

The original submission analyzes the contributions of individual components on FNDEE and evaluates the full framework on three datasets. We agree with the reviewer that examining whether the component-level trends remain consistent across datasets with different languages and domains would provide stronger evidence for the effectiveness of the framework.

Additional Experiment 2: Cross-Dataset Stage-wise Ablation with a Fixed Candidate Pool.

We conducted additional stage-wise ablations on CrudeOilNews and SENTiVENT. All expanded configurations share the same candidate pool, allowing us to distinguish the contributions of candidate expansion, eventness verification, and structural correction. The results will be included in Table 7 of the revised manuscript.

Dataset (Trig./Arg. F1)BaseExpansion+ EventnessFull
CrudeOilNews61.51/49.1060.95/48.1561.42/49.4462.12/50.75
SENTiVENT47.10/42.7946.62/42.2249.48/44.1550.38/45.69

Both datasets exhibit a consistent trend: candidate expansion improves recall but reduces F1 when used alone; eventness verification suppresses spurious candidates, while structural correction further improves the final extraction performance. These findings provide additional evidence that the contributions of individual components remain consistent across datasets.

Regarding methodological novelty, we understand the reviewer's concern about the incremental nature of the contribution. Prior work on document-level event extraction has largely addressed missed events, spurious events, and argument misassignment separately, making it difficult to balance the recall gains of candidate expansion against false-positive risks and structural conflicts. To address this limitation, our framework jointly considers candidate validity and state-conditioned action gain, allowing the recovery of missed events to trigger argument reallocation in existing records. Dynamic rescoring further enables iterative correction of multi-event structures.This design enables the joint correction of multiple interrelated error types in document-level event extraction. Experimental results demonstrate the effectiveness of the joint decision mechanism in reducing structural errors, as well as the necessity and synergistic contributions of its components.

Reviewer_kf3DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BALANCE-DEE refines the output of an existing document-level event extractor. A frozen Qwen2.5-7B-Instruct model generates extra candidate events using an enhanced schema. Trained evaluators score candidate validity, textual support, and the gains of record edits. The edit space includes insertion, deletion, and SPLIT. SPLIT adds an event while moving or sharing matched arguments from an existing record. The system executes an edit and recomputes the remaining action gains. The authors report improvements over the corresponding SEELE base extractor on FNDEE, CrudeOilNews, and SENTiVENT.

Critical strengths
  1. SPLIT provides a concrete way to change argument assignments across records. MOVE/SHARE distinguishes arguments that should be transferred from those that should remain in both records. Table 6(b) reports 68.65 argument F1 with enumerated reallocation, compared with 67.74 without SPLIT and 68.02 for Move-all. Mixed-record errors decrease from 14.8% without SPLIT to 9.6%. This supports the usefulness of the editing operation.

  2. The full system improves the corresponding base extractor. Trigger and argument F1 improve over SEELE on all three datasets. The component analyses also show that candidate expansion, eventness, and action gain affect performance. There is evidence that the pipeline works. The unresolved question is what accounts for its advantage.

Critical weaknesses
  1. The added value of the correction framework over direct LLM extraction is not convincing. Could a strong LLM perform this extraction directly? If it can achieve comparable results, the additional pipeline would be hard to justify. The paper needs a credible direct-extraction comparison to address that question. Qwen generates the extra candidates. This could already account for part of the improvement. Most of the useful work may be coming from the LLM's proposals. Tables 9 and 11 provide evidence that correction can improve over expansion alone. However, this does not settle what the full correction framework adds beyond those candidates and a competent direct extractor. The evidence should make that distinction clearer.

  2. The baseline model and its usage are unclear, making the comparison difficult to assess. The description “an LLM directly prompted with the event schema” is not enough to identify the baseline. Is it the same Qwen used for candidate generation? How is it used? These details matter directly to the comparison. LLM-Prompt achieves only 3.06 argument F1 on SENTiVENT. Model choice and usage could explain much of the gap. Without knowing the model and setup, this weak result does not convincingly establish the advantage of the correction framework. Related details are also hard to follow. Qwen is frozen and the evaluators are trained, but the construction of their training examples needs more explanation. The paper lists state perturbations without making the starting records and sampling procedure clear. The gain target is defined as the change in the average of document-level trigger and argument F1. Its predictions are mainly used for filtering and ranking. A concrete training example would help explain how that supervision is constructed and used. The textual-salience proxy also needs a clearer explanation where it is used. The main text names its inputs, but the construction and meaning of the resulting subgroup are difficult to assess from that description.

  3. More edits and dynamic rescoring change together in the iteration comparison. One-shot and Iterative execute 0.93 and 1.47 actions per document in Table 6(c). Their reported F1 values are 82.3 and 82.7. Mixed-record error rates are 13.1% and 9.6%. The iterative version performs better, but it also performs more edits. The comparison does not separate these effects. Better results do not establish that recomputing the scores is responsible for the gain. This makes the rescoring claim hard to judge.

Questions for authors
  1. What does the correction framework add over a competent direct LLM extractor? Please identify the model used for LLM-Prompt and explain how it is used. Is it the same Qwen, with the same schema information, as the candidate generator? Please clarify how much of the improvement comes from extra candidates and what the correction stage adds.

  2. How are the evaluator training examples and the salience-analysis groups constructed? Please explain the starting records, perturbations, and sampling used for evaluator training. A worked example linking a record state, candidate, edit, labels, and gain target would help. Please also explain the textual-salience proxy clearly enough in the main text to interpret the reported subgroup results.

  3. How can the rescoring effect be separated from the effect of performing more edits? Please explain the One-shot execution rules and provide a comparison with comparable editing opportunities, so the effect of rescoring can be separated from the effect of performing more edits.

5 comments
AuthorsOpenReview ↗

Response to Q1

Thank you for your careful review and constructive suggestions. We have further clarified the points raised and conducted additional experiments. The corresponding details and results will be incorporated into the revised manuscript.

(1) Compared with a direct LLM extractor, BALANCE-DEE adds candidate completion guided by discriminatively enhanced schemas, verification of event instantiation and textual support, and iterative structural correction guided by state-conditioned action gain. These mechanisms address missed events, spurious events, and cross-event argument misassignment.

(2) LLM-Prompt uses Qwen2.5-7B-Instruct, sharing the same model checkpoint as the candidate generator but using different schema information and output objectives. LLM-Prompt directly generates complete event records using the official dataset schema, whereas the candidate generator uses enhanced schemas to propose candidate events for subsequent verification and correction.

(3) Regarding the respective contributions of additional candidates and subsequent correction: For additional candidates, Table 5(b) of the original submission shows that candidate fusion increases gold-event coverage from 94.09% to 98.40%, recovering events missed by the base extractor. For subsequent correction, the new stage-wise ablation (Additional Experiment 2) shows that candidate expansion alone increases recall but reduces F1. Subsequent eventness verification and structural correction further improve trigger/argument F1 by 1.17/2.60 and 3.76/3.47 points on the two datasets, respectively.

To further address the questions concerning stronger direct extraction models and stage-wise contributions, we conducted two additional experiments.

Additional Experiment 1: Performance comparison of direct event extraction using Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct against BALANCE-DEE across three datasets.

Within each dataset, the 7B and 32B models use identical prompts, decoding settings, and output conversion procedures, with performance measured using the official evaluation protocols.

Dataset (Trig./Arg. F1)Qwen-7BQwen-32BBALANCE-DEE
FNDEE20.00/17.1932.07/24.3782.74/68.65
CrudeOilNews17.56/9.9839.63/30.3062.12/50.75
SENTiVENT9.15/3.0632.52/18.1850.38/45.69

The 32B model substantially improves direct extraction performance but remains below the full framework under the evaluated setting, highlighting the challenges of directly generating complete document-level event records.

Additional Experiment 2: Cross-dataset stage-wise ablation of candidate expansion, eventness verification, and structural correction with a fixed candidate pool.

We fix the expanded candidate pool on CrudeOilNews and SENTiVENT and evaluate the contribution of each stage.

Dataset (Trig./Arg. F1)BaseExpansion+ EventnessFull
CrudeOilNews61.51/49.1060.95/48.1561.42/49.4462.12/50.75
SENTiVENT47.10/42.7946.62/42.2249.48/44.1550.38/45.69

Both datasets exhibit a consistent trend: candidate expansion improves recall but reduces F1; eventness verification restores precision, while structural correction yields further performance gains, demonstrating the additional contribution of the correction stages.

AuthorsOpenReview ↗

Response to Q3

(1) To distinguish the contribution of dynamic rescoring from that of additional edits, we conducted two new controlled experiments: first controlling the maximum edit budget and then matching the actual number of edits. Using identical initial records and candidate pools, we compare static scoring with dynamic rescoring to assess the independent contribution of rescoring under comparable editing opportunities.

(2) Under One-shot, feasible actions are scored once against the initial record state. The strategy executes at most one highest-scoring action whose predicted gain exceeds the threshold, then terminates. Under Iterative, the record state is updated after each edit, the gains of remaining actions are recomputed, and subsequent eligible edits are selected.

To further examine whether the performance gains from dynamic rescoring persist when editing opportunities are controlled, we report the following two experiments.

Additional Experiment 3: Performance comparison of static scoring and dynamic rescoring under matched maximum edit budgets on FNDEE.

ScoringMax. editsAvg. editsTrig. F1Arg. F1Mixed Err.
Static10.9382.3068.0413.1%
Dynamic10.9382.3068.0413.1%
Static21.3182.4668.2810.6%
Dynamic21.3482.6268.489.9%

Both strategies achieve identical results under a one-edit budget. With a two-edit budget, dynamic rescoring yields additional improvements, although the average numbers of executed edits differ slightly.

Additional Experiment 4: Performance comparison of static scoring and dynamic rescoring on identical FNDEE documents with matched actual edit counts.

We further select the same 912 documents on which both strategies execute exactly two edits.

ScoringEdits#DocsTrig. F1Arg. F1Mixed Err.
Static291280.1364.7814.6%
Dynamic291280.4665.2113.4%
Difference00+0.33+0.43−1.2 pp

With the same documents and actual edit counts, dynamic rescoring still improves trigger and argument F1 while reducing mixed-record errors, indicating that its benefits cannot be fully attributed to additional edits. This conclusion is limited to the matched subset, which covers 36.5% of the test documents. Following the reviewer's suggestion, we will add these experimental settings and results to Appendix A.7.

We believe these additional experiments and methodological clarifications further establish the contributions of the individual stages and provide more direct evidence addressing the reviewer's concerns.

AuthorsOpenReview ↗

Response to Q2

(1) Evaluator training examples are constructed in four steps: initial record-state construction, error perturbation, candidate and action generation, and gold-based supervision. Salience-analysis groups are constructed using a textual-salience proxy based on trigger position, mention frequency, and discourse centrality, with the training-set median used as the partition threshold.

(2) Specifically, the evaluator training procedure is as follows.

① Starting records: We use the base extractor's original predictions on training documents and construct additional states from training-set gold event records.

② Error perturbations: Event deletion, merging same-type records, record duplication, and spurious-event insertion simulate missed events, mixed records, duplicate predictions, and spurious events, respectively.

③ Sampling: Eventness examples are drawn from Qwen proposals, base predictions, and gold records, with event-instantiation and textual-support labels assigned according to the gold events. Action-gain targets are defined as the change in the average document-level trigger and argument F1 before and after an edit (Eq. 6). We retain all positive-gain actions and sample at most 96 non-positive-gain actions per state. Gold-derived states and supplemental candidates are used only during training and development.

(3) Following the reviewer's suggestion, we added a concrete worked example in Section 3.5. The current state S=(y,p1,a1,a2)S=\\{(y,p_1,\\{a_1,a_2\\})\\} omits the gold event e2=(y,p2,a2)e_2=(y,p_2,\\{a_2\\}), with a2a_2 incorrectly assigned to the existing record. For candidate c2=(y,p2,a2)c_2=(y,p_2,\\{a_2\\}), both eventness labels are 1, while INS and SPLIT receive gain targets of 19/60 and 5/12, respectively, illustrating their different structural effects. We also added details of training-state construction and action sampling in Appendix B.1.

(4) Following the reviewer's suggestion, we will clarify the computation of the textual-salience proxy and the partitioning rules in the main text, and further elaborate on their definitions and limitations in Appendix B.2. The proxy is used only for post-hoc analysis and does not participate in model training or inference.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.