acceptodds
Under review as a conference paper at ICLR 2027

Rows Are Not Evidence Units: Evidence-Unit Invariance in Weak Temporal Localization

Abstract

Weak temporal annotations often reach the learner as table rows, but row count can change without changing the underlying supervision. A one-to-many join, selective row replication, or metadata expansion may therefore change fitted preprocessing, the relative weight of weak statements, checkpoint selection, and predictions. We define evidence-unit invariance: alternative row materializations of the same evidence should define the same learning problem. We enforce this property with quotient learning, which compiles stored rows into canonical evidence classes before preprocessing, training, and validation. Across seven temporal datasets, representing every eligible range statement by five rows changes ordinary segment F1 on all seven, with absolute changes from 0.004 to 0.228. Quotient learning leaves the clean runs unchanged and gives zero response to the same materializations. The effect persists when preprocessing is fitted once per acquisition, and separate interventions show contributions from training-risk weighting and validation-based model selection. We also evaluate six common table transformations across three dataset-model configurations. Ordinary segment F1 changes in 12 of 18 combinations and held-out probabilities in 13. Row multiplicity can therefore redefine the effective learning problem even when the underlying evidence is unchanged.

Primary area:
learning on time series and dynamical systems
Keywords:
weak supervision, temporal localization, multiple-instance learning, data preprocessing, relational learning, data-centric machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rGQgOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2(+1)
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how repeated storage of weak temporal annotations can change learning results without adding new supervision. It introduces evidence-unit invariance and proposes quotient learning, which converts stored rows into canonical evidence records before preprocessing, training, and validation. Experiments across seven temporal datasets, two model architectures, and six table transformations show that table materialization can affect feature scaling, annotation weights, checkpoint selection, and predictions under ordinary row-wise learning, whereas quotient learning remains invariant under the tested conditions.

Critical strengths

1.The paper presents an evaluation across seven temporal datasets, with additional evaluations using a second model architecture and six table transformations. This breadth provides useful evidence that the observed sensitivity is not limited to a single dataset or one duplication setting. 2. The evaluation examines predicted probabilities, model parameters, and decoded outputs in addition to segment F1. This strengthens the empirical analysis by showing that a small change in the aggregate score can hide substantial changes in the learned predictor.

Critical weaknesses
  1. The introduction does not clearly explain why the paper focuses on weak temporal annotations when duplicated rows are a general data management issue. Please clarify whether this setting introduces unique technical challenges, such as multiple heterogeneous weak statements referencing a single shared time-series acquisition, where duplication shifts intra-acquisition loss weights and inter-acquisition feature scaling, or whether it mainly serves as an application domain.
  2. Proposition 1 follows directly from the proposed preprocessing step. Once two tables are deduplicated by known identity keys and canonically sorted, a deterministic learning pipeline receives the exact same input and trivially produces zero change. While these results confirm that the implementation works as intended, it remains unclear what the mathematical formulation adds beyond grouping rows by known provenance keys (standard relational GROUP BY / SELECT DISTINCT) before running the existing learning pipeline.
  3. In all experiments, each statement is assigned an emission_id before extra rows are created, meaning the method already knows which rows come from the same statement. In existing datasets where lineage metadata is absent, Section 3.1 specifies that rows without statement or single-unit session lineage receive distinct row tokens, which leaves all duplicated rows separate and removes no duplication. The paper does not show how often such lineage is available in practice or provide a case study on an existing annotation pipeline.
  4. Although the paper notes that separate statements can have identical content, it does not report how often content-identical statements with different lineage occur in the evaluated datasets. The seven-row example in Appendix illustrates the concept but does not show its impact on the main benchmarks. Please report the frequency of content-identical statements with distinct lineage and compare the proposed method against deduplication using the same fields excluding lineage and storage metadata.
  5. The core experiment replicates only range annotations five times while keeping timestamp, positive-bag, and negative annotations at one copy. Because ordinary learning weights rows equally within each acquisition, this intervention acts as a 5x multiplier on the range loss relative to other supervision types. It is unclear how much of the reported F1 change comes from row multiplicity as a storage artifact versus changing the relative hyperparameter weight among heterogeneous loss terms.
  6. It is difficult to track the experimental settings across Sections 4 and 5, especially as the number of evaluated datasets drops from seven in Section 5.1 to four and three in Section 5.2, and two in Section 5.3. Please add a summary table showing the annotation types, duplication levels, affected splits, preprocessing variants, dataset-model combinations, seeds, and folds, and explain why specific dataset subsets and five copies of range annotations were selected.
  7. Figure 2 is crowded, and panel B's legend overlaps with panel A's x-axis label. Please increase the spacing between panels, make the legend placement consistent, define what a held-out group (nn) represents in each dataset, and state clearly that the mean and median refer to group-level changes in segment F1 rather than absolute scores. In addition, please include the clean baseline F1 scores in the main paper, so readers can contextualize the magnitude of the reported changes.
  8. In Section 5.2, applying five-copy range duplication only to the validation split improves test segment F1 by +0.111 on MHEALTH and alters held-out predicted probabilities by up to 0.879 on PPG-DaLiA, even though training data and optimization are unchanged. The paper should explain why reweighting validation range statements substantially improves test F1 on MHEALTH and whether this indicates a mismatch between the unweighted weak validation loss and the dense evaluation metric.
  9. Section 4 states that core training uses full-batch optimization, where the empirical risk and scaler statistics should be invariant to row order, yet Section 5.3 includes deterministic row reordering among the six table transformations. Please clarify whether deterministic row reordering alone alters ordinary learning under full-batch training, and if so, which part of the pipeline introduces order dependence. The paper is harder to follow than necessary given the simplicity of the underlying mechanism.
  10. Standard data management concepts (such as composite-key deduplication and canonical sorting) are presented with heavy notation and custom terminology, while key experimental baselines (such as clean F1 scores) and dataset details are deferred to the appendix. Simplifying the notation in Section 3 and consolidating the experimental setup and clean baselines in the main text would substantially improve readability.
Questions for authors
  1. Why does the paper focus specifically on weak temporal localization rather than general tabular or multi-instance learning? Is it primarily due to the many-statements-to-one-signal hierarchy in Section 3.1? Furthermore, once provenance keys are defined, what does the quotient learning formulation provide algorithmically or theoretically beyond executing a standard relational GROUP BY / SELECT DISTINCT with deterministic sorting and fitting feature scalers at the entity (acquisition) level?
  2. Because emission_id is synthetically assigned before row duplication in your experiments, and the fallback rule in Section 3.1 assigns distinct row tokens when lineage is missing (which removes no duplicates). How often is statement-level lineage actually available in existing annotation pipelines? In the evaluated datasets, how frequently do genuinely distinct statements share identical canonical support and content, and how does your method compare against a lineage-free content deduplication baseline using κ∖λ(r)=(g,a,s,τ,στ(ω),πτ(ω))\kappa_{\setminus \lambda}(r) = (g, a, s, \tau, \sigma_\tau(\omega), \pi_\tau(\omega))?
  3. Replicating only range statements five times while keeping timestamp, positive-bag, and negative statements at one copy acts as a 5×5\times multiplier on the range loss within each acquisition. How much of the reported F1 shift is caused by storage multiplicity versus changing the relative weight among heterogeneous supervision types? How does ordinary learning perform if loss terms are normalized by annotation type τ\tau, or if all statements within an acquisition are duplicated uniformly?
  4. Why does the evaluation drop from seven datasets in Section 5.1 to four and three datasets in Section 5.2, and two datasets in Section 5.3? Could you provide a summary table detailing the dataset–model combinations, splits, duplication factors, seeds, and folds across all experiments?
  5. In Section 5.2, why does applying five-copy range duplication only to the validation split improve test segment F1 by +0.111+0.111 on MHEALTH, and does this indicate a misalignment between the unweighted weak validation loss and the dense test metric? In Section 5.3, given that core training is full-batch (Section 4), does deterministic row reordering alone change ordinary learning outputs, and if so, which component of the full-batch pipeline introduces row-order dependence?
  6. Could you report the clean baseline segment-F1 scores in the main text (alongside the F1 deltas in Figure 2) so readers can judge the relative magnitude of the performance changes, and clarify what a held-out group (nn) represents for each dataset in Figure 2?
Reviewer_K6cBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a reproducibility problem in weak temporal localization: joins and repeated exports can duplicate annotation rows without adding evidence, yet change preprocessing, training, and checkpoint selection. It defines evidence identity using annotation content and lineage, then compiles rows into a canonical set of evidence units before fitting the pipeline. The resulting pipeline is invariant to evidence-preserving duplication and reordering. Experiments on six public datasets and one synthetic dataset demonstrate material changes under ordinary row-wise learning and invariance under compilation. Additional analyses separate preprocessing, training, and validation effects and establish equivalence with inverse-multiplicity weighting under matched preprocessing.

Critical strengths
  1. The paper treats duplication as a whole-pipeline problem rather than only a training-loss issue. The unique-acquisition scaling control and the training-only/validation-only experiments show that correcting one stage is insufficient. This makes the diagnosis practically useful.
  2. Evidence identity incorporates lineage instead of relying on matching numerical rows. The method preserves independent annotations with identical content while collapsing repeated materializations of the same annotation. This is an important distinction for avoiding indiscriminate deduplication.
Critical weaknesses
  1. The methodological advance is limited. Once correct evidence identities are supplied, the main guarantee follows from canonical grouping before a deterministic pipeline. Appendix J also establishes equivalence to inverse-multiplicity weighting under matched preprocessing. The experiments document a useful failure mode, but do not establish a substantial learning-method advance beyond careful provenance-aware data handling.
  2. The practical validation assumes unusually complete provenance. Weak annotations are constructed from dense labels, and statement identities are assigned before materialization. The fallback to separate row identities does not recover lost lineage. Consequently, the experiments do not establish how often the method can resolve duplication in existing weak-annotation tables, where evidence identity is the difficult part.
Questions for authors
  1. Do any evaluated tables contain naturally occurring annotation multiplicity with independently recorded lineage, rather than multiplicity introduced after assigning statement identities? If so, what proportion of rows can be assigned evidence identities without the row-identity fallback?
  2. Does canonical compilation offer a measured runtime or storage advantage over the matched inverse-multiplicity implementation in Appendix J, or is its main advantage the explicit evidence-identity contract?
Reviewer_PxTdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper identifies a data-semantics failure mode in weak temporal localization: table operations can create multiple stored rows for one underlying weak statement, causing preprocessing, empirical-risk weighting, checkpoint selection, and predictions to change even though the evidence has not. It formalizes evidence-unit invariance and proposes quotient learning. Each row is assigned an evidence key based on independent group, acquisition, source, annotation type, canonical temporal support and content, plus statement lineage, while equality of the referenced signal and validity mask is checked. Rows in the same evidence class are compiled into one canonical record before any row-dependent stage.

The paper proves exact invariance to row order and within-class copy count under deterministic execution. Empirically, it evaluates five-copy range materialization on six public datasets and one synthetic benchmark using a TCN, plus selected BiGRU experiments. Ordinary learning changes segment F1 on all seven datasets, with responses ranging from +0.004 to -0.228, whereas quotient learning is exactly invariant. Training-only and validation-only interventions show distinct pathways through risk weighting and checkpoint selection, and large probability changes can occur even when segment F1 barely moves. Six common table transformations across three dataset-model configurations alter ordinary F1 in 12 of 18 cases and probabilities in 13, while quotient learning remains invariant. Additional comparisons cover statement identity, inverse-class weighting, annotation types, multiplicities, scaler fitting, and exact parameter/output equality.

Critical strengths
  1. The paper isolates a practically important but underformalized distinction between storage identity and statistical evidence identity. The proposed invariance principle is crisp, the compiler is simple enough to implement and test, and applying it before preprocessing and validation correctly addresses failure modes that loss reweighting alone cannot. The matched-row experiment also makes clear why numerically identical statements may legitimately remain separate when their provenance differs.

  2. The empirical validation is exceptionally systematic for the claim being made. It spans seven datasets, multiple weak-statement types and multiplicities, two architectures, controlled training-versus-validation interventions, six realistic table transformations, held-group analysis, probability and decoded-output comparisons, and exact-state tests. The held-out PPG-DaLiA configuration and the zero-response quotient controls reduce concern that the headline result is an artifact of selecting a favorable dataset or metric.

Critical weaknesses
  1. The guarantee transfers the main difficulty to provenance. Correct quotienting requires reliable statement lineage and canonicalized support/content fields; missing lineage leaves duplicates separate, while erroneous shared lineage collapses independent evidence. Many real pipelines do not retain such identifiers through joins and exports. The paper acknowledges this, but does not evaluate noisy, missing, or partially recoverable lineage, so the method's practical robustness in precisely the messy settings motivating the work remains unclear.

  2. External validity is narrower than the broad framing may suggest. All seven tasks are produced within a common weak-statement construction and implementation framework, and most core results use a TCN with full-batch deterministic training. The table transformations are controlled simulations rather than audits of independently developed production pipelines. This establishes the mechanism convincingly, but not its prevalence or impact across common weak-supervision libraries, relational data systems, minibatch pipelines, or non-temporal applications.

Questions for authors
  1. How does quotient learning behave when lineage is missing, corrupted, or only probabilistically inferred? Please provide a sensitivity analysis that separately measures false merges of independent statements and missed merges of duplicated materializations.

  2. Can the authors audit at least one independently developed weak-supervision or production-style relational pipeline, where materialization artifacts arise naturally rather than through a controlled transformation? This would materially strengthen the claim of practical prevalence.

  3. When independent annotators produce numerically identical statements, what operational rule ensures that source identity and lineage preserve their distinct evidential weight? Can the compiler detect or warn about ambiguous cases instead of silently choosing row-level separation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.