- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EDIT: Given the additional evidence provided in the rebuttal, I have decided to raise my score.
__
AdaRelate formulates EEG emotion recognition as reliability-anchored sequential evidence acquisition rather than one-shot graph inference. A linear "anchor" classifier fitted on low-order, training-standardised channel descriptors supplies a zero-action logit prediction; together with a learned low-cost overview readout it forms a "prediction base" that is frozen for evaluation. A graph-constrained actor–critic then iteratively selects one legal action from a typed vocabulary: REGION (open one of 8 fixed sensor-geometry groups), CHANNEL (query one of 62 electrodes inside an opened region), RELATION (compare one of 28 opened region pairs), or STOP, invoking only the selected parameter-shared expert. Acquired evidence updates a gated GRU state (which conditions the next decision) and a bounded logit residual on top of the anchor. A strictly-forward accumulator then fuses window predictions within each trial (dataset-frozen exponential decay; cumulative mean plus uncertainty-scaled prior on MPED). Classification supervision is applied at every prefix; the actor is rewarded by the detached reduction in prefix cross-entropy minus a measured action cost.
The system is evaluated on SEED, MPED, SEED-IV, and SEED-V under personalised (within-subject) splits using 62x10x5 differential-entropy inputs, subject-level metrics (Accuracy, BAcc, macro-F1), subject-bootstrap CIs. AdaRelate exceeds the strongest reproduced mean on all 12 dataset-metric margins. Ablations on MPED attribute +7.42 pp to the fitted anchor, +6.06 pp to forward history, +1.65 pp to the uncertainty prior, and +0.55 pp to dual quality scoring. A matched three-seed MPED study gives 66.81% for Full vs 65.87% one-shot top-k (+0.93 pp) and 65.64% no-RL/static (+1.17 pp). Beyond the benchmark tables, the authors audit whether the mechanism works as intended: they zero out the acquired evidence to check whether it actually changes the policy's subsequent decisions; they replay each trajectory with a fixed number of acquisitions (0 to 6) to measure what each step adds; they replace the policy's selected targets with random or unselected alternatives of the same type; they map onto the scalp which channels, regions, and relations are used and how much each helps; and they measure inference latency and throughput.
Critical strengths
S1: The evaluation is overall rigorous. The subject, not the window or the fold, is the unit of every statistical test. Confidence intervals come from bootstrap resampling over subjects. Paired tests use 100k permutations with Holm correction. A check on 857,808 perturbed windows proves that the model never reads future data. Intervention tests that remove or replace acquired evidence measure what each component does.
S2: The formulation is genuinely novel. The model does not build the full graph in one pass. It makes a sequence of decisions: open a scalp region, query one channel in that region, compare two open regions, or stop. An action is legal only after its prerequisites. Only the selected computation runs. The model learns when to stop. This differs from one-shot adaptive-graph methods such as DGCNN and IAG/V-IAG, and from generic glimpse-selection methods. The decision path is an execution record rather than a post-hoc attribution: for each sample it shows which expert ran, on which target, and in which order.
Critical weaknesses
W1: The gains are small. When the system is forced to acquire no evidence at all (budget K0), its accuracy already equals the full system on SEED and SEED-V, is slightly better on SEED-IV, and only on MPED does acquisition help, by 0.63 points, with p = 0.1763. The gains attributed to the sequential routing policy itself are measured on MPED alone. What carries the results instead are the frozen anchor (+7.42 points) and the forward-history accumulator (+6.06 points): both are generic add-ons that could be attached to any baseline, and without the anchor the acquired-evidence system scores below a reproduced DGCNN on MPED (59.43% vs 65.75%). The remaining evidence points the same way: removing the RELATION level costs almost nothing, replacing the policy's chosen targets with random alternatives of the same type costs 0.03 points, and deleting all acquired evidence changes the next decision in only 1.45% of cases.
W2: The evaluation cannot exclude that the reported gains reflect memorisation and trial-context effects rather than emotion decoding. Every model is trained and tested on the same subject, and there is no LOSO or even fully held-out session result: the authors' own split puts "trials 1-3 from session 3" into selection training while "session-3 trials 4–15 form the outer partition" on SEED, and places "the first session-3 trial of each class" in training on SEED-IV/V (Appendix B.1). A within-subject model can therefore exploit subject- and session-specific idiosyncrasies instead of transferable emotional structure, and the protocol never tests this. Emotion labels are constant within a trial, and the forward-history accumulator, the second-largest measured effect at +6.06 points, profits directly from this persistence: once the first windows of a trial are correct, later predictions are smoothed toward the same label. Because the reproduced baselines run without the anchor and the accumulator, part of every reported margin may measure trial-context exploitation rather than better decoding.
Questions for authors
Q1: Can you report a fully held-out session-3 evaluation, and ideally a LOSO result? The model is personalised by design, but the paper motivates it through inter-individual variability, so even within the personalised setting, a clean held-out-session split is needed to show the gains reflect more than subject and session-specific patterns.
Q2: Can you report the matched three-seed routing contrasts on SEED, SEED-IV, and SEED-V, not only on MPED? Tables 7 and 8 indicate that acquisition gains are almost 0 or negative outside MPED; if confirmed, the central claim should be narrowed.
Q3: Substituting the policy's targets with random or unselected same-type alternatives costs only 0.03 points. What evidence shows that which target is selected matters, rather than merely that some legal action is taken?
A. (Suggestion) Report the accuracy and latency of a variant that runs every expert on every sample: all regions, all channels, and all relations. This gives the dense upper bound on performance and a concrete reference for the efficiency claims.
B. (Suggestion) It would be interesting if you could test the method with fewer electrodes: for example, keep only 32, 16, or 8 of the 62 channels, or even use a low-cost EEG dataset with 4 channels. If sequential acquisition is what drives the improvement, its advantage should grow as the number of possible targets grows; if it stays near zero at every channel count, the gains come from the prediction base. Low-cost devices also produce noisier channels, which is the setting where the channel-reliability scoring should help most.