acceptodds
Under review as a conference paper at ICLR 2027

Structured Semantic Evidence for Managing Uncertainty under Sparse ASL Sensing

Abstract

Perception-driven communication systems often operate under uncertain evidence because sensing can be incomplete and recognition models can make errors. A single predicted sequence or confidence score therefore cannot reveal which parts of the intended meaning are actually supported. We frame this as a context-conditioned semantic evidence problem: given Top-K recognition hypotheses, their relative probabilities, and dialogue context, infer which semantic information is required, determined, uncertain, or missing in the current conversational turn. We introduce a slot-specific cross-attention model that jointly reasons over recognition hypotheses and context to construct a structured semantic evidence representation: a collection of per-slot records indicating which meaning dimensions are required and whether their current evidence is Determined, Uncertain, Missing, or NotRequired. Experiments on ASL as a challenging testbed show that both recognition evidence and dialogue context make complementary contributions to structured semantic evidence estimation, while poorer recognition quality is consistently associated with weaker semantic recovery. Controlled context interventions further improve English meaning recovery when the correct conversational context is available. These results support structured semantic evidence as an intermediate representation between uncertain recognition and downstream conversational reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM agents, sign language recognition, uncertainty-aware decision making, dialogue clarification and repair, expected value of information, assistive technology

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_X1ohOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a structured semantic evidence framework for uncertainty management in American Sign Language (ASL) communication. Instead of relying on sentence-level recognition confidence, the method uses Top-K recognition hypotheses and dialogue context to predict whether individual semantic slots are determined, uncertain, missing, or not required. The authors introduce a slot-specific cross-attention model and a human-annotated benchmark for evaluating semantic evidence and communication safety. On unseen signers, the proposed model achieves 0.660 evidence-state macro-F1 and 0.927 repair AUROC, outperforming several input ablations and a zero-shot GPT-5.5 baseline. Additional experiments examine recognition quality, contextual information, and downstream repair decisions.

Critical strengths
  1. A practical formulation of uncertainty beyond sentence-level confidence. The paper addresses an important problem: a recognition system may be confident in its prediction while missing meaning-critical information such as negation or time. Representing uncertainty at the semantic-slot level is useful because it identifies what information is unresolved rather than simply flagging an entire utterance as unreliable.

  2. Evidence that recognition hypotheses and dialogue context provide complementary information. The ablation results show that combining recognition hypotheses with dialogue context improves evidence-state macro-F1 from 0.502 and 0.521 to 0.660. The signer-disjoint evaluation also provides some evidence that the method generalizes beyond the training signers.

Critical weaknesses
  1. The paper does not demonstrate that structured semantic evidence improves downstream communication. The central motivation is to make ASL communication safer by identifying and repairing uncertain meanings. However, the main results only demonstrate improvements in slot-level classification. English meaning recovery is evaluated separately using GPT-5.5, and the proposed structured representation is not shown to improve translation or communication safety over simpler alternatives. More importantly, the repair policy achieves only 7.35% agreement with human actions, and no actual post-repair recovery is evaluated. As a result, the paper does not establish whether its central contribution provides a meaningful benefit beyond predicting semantic-slot labels.

  2. Limited methodological novelty and insufficient comparison with closely related approaches. The proposed architecture largely combines established components: slot-specific attention from dialogue state tracking (Yang et al., 2023), semantic uncertainty over multiple hypotheses (Kuhn et al., 2023; Farquhar et al., 2024), and supervised classification of semantic evidence states. While their combination is reasonable, it is unclear what fundamentally new modeling capability is introduced. The baselines mainly remove input components or compare against zero-shot GPT-5.5. Without comparisons to simpler supervised slot classifiers or existing structured uncertainty methods, the experiments do not establish whether the proposed cross-attention architecture is necessary.

References mentioned

  • Yang et al. (2023). Dialogue State Tracking with Sparse Local Slot Attention. NLP4ConvAI.
  • Kuhn et al. (2023). Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation.
  • Farquhar et al. (2024). Detecting Hallucinations in Large Language Models Using Semantic Entropy. Nature.
Questions for authors
  1. Can the authors demonstrate that the proposed structured semantic evidence improves downstream release or repair decisions compared with simpler slot-level confidence estimates? This would directly test the paper's main motivation.

  2. How does the proposed model compare against a standard supervised slot classifier using the same Top-K hypotheses and dialogue context, but without the proposed slot-specific cross-attention mechanism? This comparison is important for establishing the architectural contribution.

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We appreciate the reviewer's focused questions regarding downstream benefit and the necessity of slot-specific cross-attention.

1. Downstream benefit over simpler confidence estimates. The complete structured system clearly outperforms a scalar-confidence policy. To isolate the slots themselves, we also compared it with an otherwise identical system that has the same Top-K and dialogue context but no slots; the structured condition passes only per-slot metadata, no slot content. 95% cluster-bootstrap CIs over reference sentences.

EvidenceMetricBeforeAfterΔ [95% CI]
vs. scalar-confidence policyAcceptable action47.14%58.16%+11.02 pp [+7.8, +14.3]
Target-slot F149.19%82.08%+32.89 pp
Excess requested slots / utt.4.276.409−90.4%
Slots only (vs. no-slot)Negation preserved (417 negated utt.)52.8%70.7%+18.0 pp [+13.7, +22.4], McNemar p = 1e-21
Required-slot F1.235.252+.017 [+.010, +.023]
Target-slot F1.802.821+.019 [+.006, +.032]
Unnecessary requested slots / utt..454.409−.045 [−.073, −.016]
vs. original heuristicExact / acceptable action7.35 / 11.17%41.5 / 58.2%+34.1 / +47.0 pp
Post-repair recovery (simulated)Req-slot F1, no repair → repair.252.584+.333 [+.313, +.352]
Req-slot F1, random → our questions.408.584+.176 [+.159, +.193]
Safer releaseGuardian unsafe release @ 3.89%80.00%67.27%−12.73 pp [−25.6, −1.0]

2. Matched supervised classifier without cross-attention. We added the requested supervised baseline. It receives exactly the same Top-K glosses, probabilities, partner context, and slot identity, but replaces slot-specific cross-attention with probability-weighted mean pooling and a shared MLP.

ModelState macro-F1State accuracy
Traditional slot-confidence classifier (mean of 5 seeds).506—
GPT-5.5 8-shot.528.623
Shared pooling + MLP (same inputs).578.674
Slot-specific cross-attention (ours)*.681 (+.103, +17.9% rel.).761

All five seed-wise differences are positive, isolating the architecture from input information. *Ours and shared pooling are retrained in one pipeline (5 seeds); Table 1 reports .660 ± .028 (ablations −.139 / −.158) from the original GPU run.

3. Novelty: semantic evidence sufficiency under uncertain perception.

Beyond prediction uncertainty. Uncertainty estimation measures disagreement among predictions (Kuhn et al., 2023; Farquhar et al., 2024), but agreement does not imply semantic completeness. A signer may mean "I do not want to cancel," while every Top-K hypothesis reads WANT CANCEL APPOINTMENT: semantic entropy is low, yet the negation is lost. We model not only what the recognizer predicts, but whether its predictions sufficiently support the information the turn requires.

Formulation. For each slot, the model jointly decides (i) whether the interaction requires it and (ii) whether the hypotheses support it: Determined, Uncertain (competing evidence), Missing (required but unsupported, even when hypotheses agree), or NotRequired. Unlike slot tracking (Yang et al., 2023), we assess the sufficiency of imperfect perceptual evidence rather than extract values from clean text.

Architecture. Slot-specific queries attend separately to the hypotheses and the context, with gated fusion, so each slot compares what the interaction requires against what the recognizer supports (shared pooling instead: .681 → .578; removing hypotheses / context: −.130 / −.125 State F1).

Comparison and Ablation. Each state maps to an action: retain what is Determined, clarify what is Uncertain, request what is Missing, ignore what is NotRequired; #1 shows this improves release and targeted clarification.

Utterance-level uncertainty (Kuhn et al., 2023; Farquhar et al., 2024)Dialogue state tracking (Yang et al., 2023)Ours: semantic evidence sufficiency
Unit of uncertaintyOne score per utteranceSlot values, no evidence stateEach slot: Determined / Uncertain / Missing / NotRequired
Evidence sourceOwn sampled outputsClean text dialogue historyNoisy Top-K perception + probabilities + current turn
Hypotheses agree but miss a negationLow entropy → releasedText assumed correctContext requires polarity → flagged unresolved
Downstream actionRelease all or repeat allFill slot valuesRelease / clarify / request / ignore per slot
State F1 on our Test (same split).462 (slot-level semantic entropy).559 (text-only dialogue + 1-best).681

The anonymized repository includes the matched baseline code, five-seed results, per-sample outputs, all downstream analyses, and verification artifacts.

Dear Reviewer,

Thank you again for your thoughtful review.

As the author–reviewer discussion period is drawing to a close and the deadline is only a few hours away, we would greatly appreciate your further feedback regarding whether our rebuttal has adequately addressed your concerns.

If there are still any remaining issues or questions, please give us a chance to clarify them before the discussion period ends.

Thanks a lot for your valuable time and hard work.

Reviewer_njfcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article changes the identification uncertainty in ASL communication from "giving a total confidence level" to judging by semantic slot: which information is necessary, determined, still uncertain, or missing. The main contributions are: formalizing the problem, proposing a slot specific cross attention model that integrates Top-K recognition assumptions and dialogue context, and constructing a designer disjoint manually annotated communication security benchmark. In terms of verification, when no designer is seen, both will significantly drop points if one is missing. Removing the recognition hypothesis/context will result in a decrease of 0.139/0.158 in state macro-F1, respectively; A lower WER subset is better; The correct context improves the English reconstruction of Token-F1 by approximately 0.038; Global risk and slot by slot evidence complement each other, while "identifying uncertainty" and "how to fix it" are separable issues.

Critical strengths

One is to reconstruct ASL communication uncertainty into slot by slot semantic evidence, clearly distinguishing between necessary, certain, uncertain, and missing, which is more informative than a single confidence level and provides an interpretable interface for recognition and downstream repair; This makes the paper not just an engineering assembly, but a clear problem statement. The second is that the experiment is more targeted: removing both recognition hypotheses and context at the same time without a designer will significantly drop points, and controlling for context intervention also supports the complementarity of the two.

Critical weaknesses

Firstly, the downstream action prototype is too weak: the repair recall is high, but the release coverage is only 3.89%, which is only 7.35% completely consistent with manual actions, and only 11.17% with acceptable alternatives. There is also no evaluation of post repair recovery, delay, and signer burden, making it difficult to support the system contribution of "communication security interaction". Secondly, the safety benchmark evidence is relatively weak: single person annotation, no frame level grounding, 94.63% of the Test set is unsafe, the indicators are prone to being optimistic, and the code corpus is not publicly available.

Questions for authors
  1. Test set 94.63% unsafe, can we supplement the AUROC/AUC/ECE and release coverage under natural pre valence or balanced subsets? 2) What is the release coverage and exact agreement for A0-A7, which are relatively random, majority class, and only have a global risk baseline? 3) Is there consistency in the subset of second annotators for individual annotation? 4) Can you provide anonymous code and desensitization samples to reproduce Table 1/2?
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for highlighting actionability, class imbalance, annotation quality, and reproducibility.

1. Downstream actionability and signer burden. We added learned-policy and localization evaluations. 95% cluster-bootstrap CIs over reference sentences.

ComparisonMetricBaselineStructuredΔ [95% CI]
vs. scalar-confidence policyAcceptable action47.14%58.16%+11.02 pp [+7.8, +14.3]
Target-slot F149.19%82.08%+32.89 pp
Excess requested slots / utt.4.276.409−90.4%
vs. original A0–A7 heuristicExact / acceptable action7.35% / 11.17%41.48% / 58.16%+34.1 / +47.0 pp
vs. identical no-slot systemTarget-slot F1.802.821+.019 [+.006, +.032]
Unnecessary requested slots / utt..454.409−.045 [−.073, −.016]
Offline repair replayQuestions / utt. (same completion rate)2.9422.900−.043 [−.083, −.003]
Gold slots recovered / question.852.865+.013 [+.004, +.021]

Slots therefore lower signer burden: fewer questions, each of which recovers more of the missing meaning; since questions drive interaction delay, the question count serves as our offline delay proxy. Post-repair recovery: when the asked slots are answered (simulated with the annotated slot evidence), required-slot F1 rises from .252 to .584 (+.333 [+.313, +.352]); our 2.84 questions per utterance recover 85% of the gain of asking all seven slots, i.e., 2.1× more meaning per question. Q2 (all exact/acceptable differences vs. structured exclude 0):

MethodRelease cov.Unsafe releaseExactAcceptable
Random (Dev prior)9.3%94.6%16.6%32.0%
Majority (A4)0%—17.4%23.0%
Global risk only35.4%88.4%12.2%15.3%
Structured policy6.1%64.4%41.5%58.2%

2. Safety imbalance and matched coverage. We added a strict signer-OOF guardian and report the class composition explicitly (76 SAFE, 1,339 UNSAFE; 94.63% unsafe prevalence). All systems below release the same number of utterances.

SystemMetricWithout slotsWith slotsΔ [95% CI]
GuardianAUROC / AURC.7923 / .8858.8117 / .8746better on both
Unsafe release @ 3.89% (55 utt.)80.00%67.27%−12.73 pp [−25.6, −1.0]
Unsafe release @ 6.15% (87 utt.)79.3%71.3%−8.1 pp [−16.9, −0.2]
A0–A7 policyRelease ranking AUROC.800.814+.015 [+.002, +.030]

The gain is not specific to one operating point. Our A0–A7 policy is safer still: at 3.89% coverage it releases 50.9% unsafe utterances, −16.4 pp [−33.1, −1.2] vs. the guardian with per-slot probabilities. Q1: AUROC is prevalence-invariant. Re-weighting Test to the Development prevalence and to 50% (the expectation over balanced 76 + 76 subsets), with prior-shift-corrected probabilities (ECE is for the guardian's signer-OOF Platt-calibrated output; Table 4's .294 is the uncalibrated fusion):

Unsafe prev.AURCBrierECEUnsafe release @ 3.89%Dev threshold: release cov. / unsafe release
94.6% (Test).886 → .875.048 → .046.023 / .02380.0 → 67.3%, −12.7 pp [−25.6, −1.0]35.4 → 21.9% / 88.4 → 84.8%
90.7% (Dev).810 → .794.076 → .073.042 → .03968.8 → 53.2%, −15.7 pp [−30.2, −1.4]37.1 → 23.6% / 80.8 → 75.5%
50% (balanced).287 → .267.194 → .190.102 / .10418.5 → 10.4%, −8.1 pp [−19.0, −0.9]54.7 → 40.7% / 30.2 → 24.1%

At the Development-selected threshold, per-slot evidence lowers unsafe release at every prevalence (all Δ CIs exclude 0).

3. Annotation consistency and frame grounding. We added an independent second annotation of 160 test utterances, stratified by gold action (20 per class). The two annotators agree substantially on the release-versus-repair decision (κ = .71). On slot states, our model's agreement with the independent annotator (α = .43) is close to human–human agreement (α = .46), so the model approaches the human ceiling of this annotation. We also clarify that the benchmark evaluates turn-level semantic evidence over recognition hypotheses and dialogue context, rather than frame-level temporal grounding.

4. Reproducibility. The raw egocentric video corpus is under review in a separate submission. We will release the Stage 1 recognition outputs together with the reference glosses and English translations, and all subsequent code, data, and annotations. The anonymized repository includes de-identified examples, the complete Table 1/2 pipeline, the added reviewer baselines, strict signer-OOF per-sample guardian predictions, fixed checkpoints, metric definitions, the second annotation, all downstream analyses with per-sample outputs and bootstrap scripts, and independent verification.

Dear Reviewers,

Thank you again for your thoughtful review.

As the author–reviewer discussion period is drawing to a close and the deadline is only a few hours away, we would greatly appreciate your further feedback regarding whether our rebuttal has adequately addressed your concerns.

If there are still any remaining issues or questions, please give us a chance to clarify them before the discussion period ends.

Thanks a lot for your valuable time and hard work.

I read the rebuttal and revised paper. The main concerns remain unresolved. I keep my original score and decision.

Thank you for reviewing our rebuttal and revised manuscript. We would like to briefly clarify how we addressed your concerns:

  1. Downstream actionability: We replaced the original A0–A7 heuristic with a learned repair policy and added simulated post-repair evaluation. Exact human-action agreement improved from 7.35% to 41.48%, acceptable action agreement improved from 11.17% to 58.16%, and post-repair required-slot F1 increased from 0.252 to 0.584. Structured evidence also improved targeted clarification.

    Regarding the concern of communication-safety interaction and scope of this work: we agree that downstream validation is important and have demonstrated measurable improvements. However, our primary contribution is structured semantic evidence estimation and a communication-safety benchmark, rather than a complete interactive system. Such a system requires integration with upstream ASL recognition, real-time clarification, and evaluation with Deaf signers. We will also explicitly acknowledge the lack of fully integrated, real-world interaction evaluation as a limitation in the revised manuscript.

  2. Safety and class imbalance: We added matched-coverage and prevalence-adjusted evaluations, reducing unsafe releases from 80.0% to 67.3% at 3.89% coverage.

  3. Annotation reliability: We added independent annotations for 160 utterances, obtaining κ=0.71\kappa = 0.71 for release-versus-repair decisions.

  4. Reproducibility: We provided an anonymized repository with code, annotations, additional baselines, and evaluation results.

If any specific concerns remain, we would appreciate your feedback and would be happy to provide further clarification.

Thank you again for your time and consideration.

Reviewer_SNLeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper's main focus is to identify parts of a sentence, in perceptive settings like ASL, which are uncertain or unclear or if the sentence is incomplete or incorrect. To do so, the authors create a fixed set of 7 fields, which they term as semantic slots and they include categories like person, time, negation and action etc. They train a slot-specific criss-attention model to classify each slot as determined, uncertain, missing or not required based on the context of the conversation and recognition hypotheses. They test the model on a human-labeled ASL benchmark but actually do generalization studies on unseen signers. They show that this method outperforms the baselines which include considering input only and a GPT-5.5 under zero-shot settngs. While the paper also tries to calrify or "repair" the evidence of that working is rather weak or is actually limited to considering offline agreement with human labels.

Critical strengths

The paper is well-written, well-motivated and easy to follow. The problem formulation and its goals are clear and having a technique that can identify which parts of a message are uncertain or if the entire perceived message is wrong seems to be a very important problem. Furthermore, identifying these can enable targeted clarification rather than just asking to repeat the entire message. So having a granular score rather than at the sentence level limits actionability.

Under a restricted scenario, having structured fields makes a lot of sense in parsing the messages. Under that assumption the approach is convincing. The ablations also show that the recognition hypotheses and the dialogue context contribute complemetary information. The method also generalizes to unseen signers.

Critical weaknesses
  1. While the motivation as I noted is compelling, the repair section results are not convincing. If a system knows exactly which parts of the message is uncertain or missing, it should be then able to ask a targeted question and recover the meaning. However, the experiments show that the model can mainly predict the evidence states but not that they increase the clarity or more accurately. The reconstruction experiment shouws that providing correct context helps the GPT, but it does not compare reconstruction with and without learned slots. Consequently the downstream benefits are unclear. Also, due to the limitations of the dataset chosen, the intervention benefits are not established. Consequently an evaluation of recovered meaning would address this gap. I acknowledge that authors list this in their limitations section and I appreciate that but this concern is big enough that the authors should address this before publication.

  2. The paper feeds to the downstream tasks uncalibrated weights, (lines 103), which seems reasonable but then the downstream benefits are not clear if the model is using these weights or is it mostly having a number of sequences to pattern-match against. So a simple test could be to shuffle or intriduce noise in these weights while keeping the sets constant to see if the model is actually using these weights/confidences at all.

  3. It is not clear if the GPT-5.5 zero-shot baseline is a strong enough baseline. Why zero-shot? May be if the authors provide few in-context examples, the GPT-5.5 performance could improve as well. Therefore, I request the authros to do a few-shot ICL example based GPT-5.5 analysis and report its performance.

  4. Lastly, the lexicon is hand-written and it is unclear if the model's performance is robust to noisy construction which could be the case as the system gets expaned. So similar to 2, can the authors conduct a small experiment with varying levels of noise in the mapping and see the models performance. ?

Questions for authors

Please weaknesses.

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for the constructive questions about downstream benefit, recognition weights, few-shot LLM baselines, and lexicon robustness.

1. Downstream benefit. We added downstream analyses on all 1,415 test utterances. Together they form one chain of evidence: slots preserve critical meaning, a controlled slot intervention shows that better slot evidence yields better meaning, slots tell the system when and what to ask, and answering those questions recovers the meaning. Before → After = no-slot → structured in (a) and (c), 0% → 100% gold slots‡ in (b), and no repair → simulated targeted repair§ in (d). No-slot and structured share the same Top-K, context and prompt; the structured condition gives GPT-5.5 only per-slot metadata (required / state / repair probabilities) and no slot content. 95% cluster-bootstrap CIs over reference sentences (B = 5,000).

StepMetricBeforeAfterΔ [95% CI]
(a) Meaning preservedNegation preserved (417 negated utt.)52.8%70.7%+18.0 pp [+13.7, +22.4], McNemar p = 1e-21
Polarity error (all)22.8%18.1%−4.7 pp [−6.6, −3.0]
Required-slot F1.235.252+.017 [+.010, +.023]
Omitted object / person (LLM judge†)28.8% / 17.5%24.3% / 14.3%−4.5 / −3.3 pp
(b) Controlled slot interventionRequired-slot F1, 0 → 100% gold slots (dose–response).252.642slope +.393 [+.373, +.412], ρ = .97
Polarity error, gold negation slot only18.1%9.8%−8.3 pp [−10.5, −6.2]
(c) When / what to askReleased req-slot F1 @ 20% coverage.378.446+.068 [+.033, +.096]
Target-slot F1.802.821+.019 [+.006, +.032]
Unnecessary requested slots / utt..454.409−.045 [−.073, −.016]
(d) Repair recovers meaningRequired-slot F1.252.584+.333 [+.313, +.352]
Polarity error18.1%11.2%−6.9 pp [−8.7, −5.1]
Req-slot F1, random → our questions (same number).408.584+.176 [+.159, +.193]

†Validated on 100 human-labelled pairs (recall .98 for critical errors). ‡Gold slots: for each of the 9,905 Test (utterance, slot) pairs (7 slots per utterance), the human label of whether the slot is required, its evidence state, the supporting gloss content (e.g., LIKE LOOK; for negation, only a polarity mark), and whether repair is needed. Only this oracle condition gives GPT-5.5 this content. §For the slots our policy asks about (2.84 per utterance), the gold slot is inserted as the signer's answer and GPT-5.5 reconstructed again. Asking all seven slots reaches .642, so targeted questions recover 85% of the attainable gain with 41% of the questions.

The complete structured system also outperforms a scalar-confidence policy: Target-slot F1 49.19% → 82.08% (+32.89 pp) and mean excess requested slots 4.276 → .409 (−90.4%).

2. Use of recognition weights. We froze all five checkpoints and changed only the Top-K probabilities while preserving candidate identity and order. Uniform, rank-only, temperature-scaled, shuffled, and noisy probabilities change State F1 by at most .0016. The model is therefore robust to miscalibrated recognizer scores: its slot evidence is driven by gloss identity and dialogue context. This is a useful property under sparse sensing, where recognizer confidences are unreliable. We will describe this precisely in the revision.

3. Few-shot GPT-5.5. We added GPT-5.5 baselines using signer-disjoint, Development-only demonstrations.

ModelState macro-F1Repair AUROC
GPT-5.5 zero-shot.503.808
GPT-5.5 4-shot.510.806
GPT-5.5 8-shot.528.809
Ours (slot-specific cross-attention)*.681 (+.153, +29.0% rel.).930 (+.121, +15.0% rel.)

*Retrained in one pipeline used for all model comparisons (5 seeds): .681 ± .017 vs. .660 ± .028 in Table 1 (original GPU run), within seed variation. We will report both.

4. Lexicon robustness. The selected cross-attention model does not query the hand-written lexicon at inference, so lexicon errors cannot affect it. The lexicon belongs only to the traditional slot-confidence baseline, on which we tested perturbations:

ModelConditionState F1
Traditional baseline (uses lexicon; 5-seed ensemble)Clean.509
Delete 30% of anchors.469
Reassign 20% across slots.475
Add distractorslittle effect
Ours (lexicon-free)—.681 (+.172 over clean baseline)

We will make this separation explicit.

The anonymized repository https://anonymous.4open.science/r/SparseASLagent/ includes the new supervised baselines, 4-/8-shot GPT evaluation, probability and lexicon perturbation scripts, all downstream analyses with per-sample outputs and bootstrap scripts, and verification files.

Dear Reviewer,

Thank you again for your thoughtful review.

As the author–reviewer discussion period is drawing to a close and the deadline is only a few hours away, we would greatly appreciate your further feedback regarding whether our rebuttal has adequately addressed your concerns.

If there are still any remaining issues or questions, please give us a chance to clarify them before the discussion period ends.

Thanks a lot for your valuable time and hard work.

Best

Reviewer_SNLeOpenReview ↗

Thank you for additional experiments

Thank you for additional experiments and insights. Authors have demonstrated robustness of their approach w.r.t to the recognition weights and lexicon robustness. It is very surprising to see GPT-5.5 not registering any improvement in performance with few-shot examples over zero-shot. Authors should have included why they think this happened. I will increase my score to 3 and will discuss with other reviewers in the discussion phase.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for the helpful observation and for reconsidering the score. We have added an explanation in Section 4.3: few-shot examples clarify the task and label semantics, but our supervised model explicitly learns how multiple noisy recognition hypotheses and dialogue context jointly support each required semantic slot. We appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.