- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2(+1)
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how repeated storage of weak temporal annotations can change learning results without adding new supervision. It introduces evidence-unit invariance and proposes quotient learning, which converts stored rows into canonical evidence records before preprocessing, training, and validation. Experiments across seven temporal datasets, two model architectures, and six table transformations show that table materialization can affect feature scaling, annotation weights, checkpoint selection, and predictions under ordinary row-wise learning, whereas quotient learning remains invariant under the tested conditions.
Critical strengths
1.The paper presents an evaluation across seven temporal datasets, with additional evaluations using a second model architecture and six table transformations. This breadth provides useful evidence that the observed sensitivity is not limited to a single dataset or one duplication setting. 2. The evaluation examines predicted probabilities, model parameters, and decoded outputs in addition to segment F1. This strengthens the empirical analysis by showing that a small change in the aggregate score can hide substantial changes in the learned predictor.
Critical weaknesses
- The introduction does not clearly explain why the paper focuses on weak temporal annotations when duplicated rows are a general data management issue. Please clarify whether this setting introduces unique technical challenges, such as multiple heterogeneous weak statements referencing a single shared time-series acquisition, where duplication shifts intra-acquisition loss weights and inter-acquisition feature scaling, or whether it mainly serves as an application domain.
- Proposition 1 follows directly from the proposed preprocessing step. Once two tables are deduplicated by known identity keys and canonically sorted, a deterministic learning pipeline receives the exact same input and trivially produces zero change. While these results confirm that the implementation works as intended, it remains unclear what the mathematical formulation adds beyond grouping rows by known provenance keys (standard relational GROUP BY / SELECT DISTINCT) before running the existing learning pipeline.
- In all experiments, each statement is assigned an emission_id before extra rows are created, meaning the method already knows which rows come from the same statement. In existing datasets where lineage metadata is absent, Section 3.1 specifies that rows without statement or single-unit session lineage receive distinct row tokens, which leaves all duplicated rows separate and removes no duplication. The paper does not show how often such lineage is available in practice or provide a case study on an existing annotation pipeline.
- Although the paper notes that separate statements can have identical content, it does not report how often content-identical statements with different lineage occur in the evaluated datasets. The seven-row example in Appendix illustrates the concept but does not show its impact on the main benchmarks. Please report the frequency of content-identical statements with distinct lineage and compare the proposed method against deduplication using the same fields excluding lineage and storage metadata.
- The core experiment replicates only range annotations five times while keeping timestamp, positive-bag, and negative annotations at one copy. Because ordinary learning weights rows equally within each acquisition, this intervention acts as a 5x multiplier on the range loss relative to other supervision types. It is unclear how much of the reported F1 change comes from row multiplicity as a storage artifact versus changing the relative hyperparameter weight among heterogeneous loss terms.
- It is difficult to track the experimental settings across Sections 4 and 5, especially as the number of evaluated datasets drops from seven in Section 5.1 to four and three in Section 5.2, and two in Section 5.3. Please add a summary table showing the annotation types, duplication levels, affected splits, preprocessing variants, dataset-model combinations, seeds, and folds, and explain why specific dataset subsets and five copies of range annotations were selected.
- Figure 2 is crowded, and panel B's legend overlaps with panel A's x-axis label. Please increase the spacing between panels, make the legend placement consistent, define what a held-out group () represents in each dataset, and state clearly that the mean and median refer to group-level changes in segment F1 rather than absolute scores. In addition, please include the clean baseline F1 scores in the main paper, so readers can contextualize the magnitude of the reported changes.
- In Section 5.2, applying five-copy range duplication only to the validation split improves test segment F1 by +0.111 on MHEALTH and alters held-out predicted probabilities by up to 0.879 on PPG-DaLiA, even though training data and optimization are unchanged. The paper should explain why reweighting validation range statements substantially improves test F1 on MHEALTH and whether this indicates a mismatch between the unweighted weak validation loss and the dense evaluation metric.
- Section 4 states that core training uses full-batch optimization, where the empirical risk and scaler statistics should be invariant to row order, yet Section 5.3 includes deterministic row reordering among the six table transformations. Please clarify whether deterministic row reordering alone alters ordinary learning under full-batch training, and if so, which part of the pipeline introduces order dependence. The paper is harder to follow than necessary given the simplicity of the underlying mechanism.
- Standard data management concepts (such as composite-key deduplication and canonical sorting) are presented with heavy notation and custom terminology, while key experimental baselines (such as clean F1 scores) and dataset details are deferred to the appendix. Simplifying the notation in Section 3 and consolidating the experimental setup and clean baselines in the main text would substantially improve readability.
Questions for authors
- Why does the paper focus specifically on weak temporal localization rather than general tabular or multi-instance learning? Is it primarily due to the many-statements-to-one-signal hierarchy in Section 3.1? Furthermore, once provenance keys are defined, what does the quotient learning formulation provide algorithmically or theoretically beyond executing a standard relational GROUP BY / SELECT DISTINCT with deterministic sorting and fitting feature scalers at the entity (acquisition) level?
- Because emission_id is synthetically assigned before row duplication in your experiments, and the fallback rule in Section 3.1 assigns distinct row tokens when lineage is missing (which removes no duplicates). How often is statement-level lineage actually available in existing annotation pipelines? In the evaluated datasets, how frequently do genuinely distinct statements share identical canonical support and content, and how does your method compare against a lineage-free content deduplication baseline using ?
- Replicating only range statements five times while keeping timestamp, positive-bag, and negative statements at one copy acts as a multiplier on the range loss within each acquisition. How much of the reported F1 shift is caused by storage multiplicity versus changing the relative weight among heterogeneous supervision types? How does ordinary learning perform if loss terms are normalized by annotation type , or if all statements within an acquisition are duplicated uniformly?
- Why does the evaluation drop from seven datasets in Section 5.1 to four and three datasets in Section 5.2, and two datasets in Section 5.3? Could you provide a summary table detailing the dataset–model combinations, splits, duplication factors, seeds, and folds across all experiments?
- In Section 5.2, why does applying five-copy range duplication only to the validation split improve test segment F1 by on MHEALTH, and does this indicate a misalignment between the unweighted weak validation loss and the dense test metric? In Section 5.3, given that core training is full-batch (Section 4), does deterministic row reordering alone change ordinary learning outputs, and if so, which component of the full-batch pipeline introduces row-order dependence?
- Could you report the clean baseline segment-F1 scores in the main text (alongside the F1 deltas in Figure 2) so readers can judge the relative magnitude of the performance changes, and clarify what a held-out group () represents for each dataset in Figure 2?