- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the mismatch between permutation symmetries and the symmetries of sets, specifically the multiplicity of elements in contrast to multisets. The authors argue that permutation invariance is insufficient when true sets are desired (unordered and multiplicity invariant) and propose definitions to characterize this notion as idempotent set consistency. The extent to which several existing methods are sensitive to (near) duplicate insertion is measured in three experiments.
Critical strengths
Interesting problem
I think this is an interesting problem to study. Similar to how there are subtleties between the interaction of permutation symmetry and multisets (multiple copies allowed, the authors make a case that the same is true for permutation symmetry and sets (multiple copies disallowed) and I would be inclined to agree. The study of how exactly the multiplicity affects how sets should be modeled is interesting purely from a theoretical level alone, and the paper makes a good start in characterizing the problem, while also touching on the important aspect that continuity should be respected.
This point is consequential because I believe that the the topic itself would be of interest to ICLR audiences.
Critical weaknesses
Insufficient depth
I was disappointed to see a lack of insight into how the formalization can be used beyond a simple evaluation of the proposed metrics on trained models. As it stands, it appears to be an empirical metric that cannot be used a-priori to judge whether a model architecture is suited for satisfying the properties the authors define. It is presented as a purely post-hoc analysis of how robust some models are to different multiplicities, making it significantly less applicable to practical scenarios.
There does not seem to be any attempt made at discussing whether the stability to duplicate elements can be guaranteed through architecture. Such a discussion is usually a core aspect of papers in the field. Given the inherently discontinuous objective here, achieving it seems tricky, so this is a major omission in my view.
The main conclusion seems to be essentially "you should measure to what extent models are sensitive to duplicates if you care about duplicates", which is a fairly obvious statement. I did not get any larger insights than that from my reading of the paper.
This point is consequential in that in my opinion, the lack of depth of the study makes it insufficient to clear the bar of an ICLR submission. It feels more appropriate as a submission to a workshop, and after the ideas in the paper (which are a good start) have been fleshed out further, it might qualify for a conference in the future.
Questions for authors
My main question is in terms of any real-world application of the proposed method:
- To what extent are duplicate observations realistic? With the example in the paper: duplicate pedestrian registration is only duplicate if all identifying information is exactly equal too (i.e. position, size, but also contents of registered pedestrian are equal). It seems easy to just pre-process such cases away if they are undesirable. Why should we rely on the model for that? With higher dimensionality of set elements it also seems less likely for elements to end up exactly equal.
6 comments
Response to Reviewer pvkZ (2/2): Practical relevance and near-duplicate forecasts
To what extent are duplicate observations realistic?
Duplicate detections and tracks remain a documented problem in modern tracking. Beyond UMOT's comparisons with MOTR and MOTRv2 already cited in §5.3, two relevant examples are:
- MCTR, WACV 2025 Workshops, §4 reports that MOTR, evaluated independently in each camera view, produces highly overlapping duplicate detections and consequently duplicate tracks. Its authors add NMS with an IoU threshold of 0.95 to address this observed failure.
- TrackTrack, CVPR 2025 introduces Track-Aware Initialization specifically to suppress new tracks from detections that overlap active tracks or more confident detections.
The first documents an actual duplication failure; the second shows that preventing redundant track initialization remains an active design problem in a recent tracker.
duplicate pedestrian registration is only duplicate if all identifying information is exactly equal too
The relevant redundancy is that multiple records refer to the same physical pedestrian. Their estimated positions or histories need not be numerically identical. Our audited inputs are eight-step position histories, and the inserted records have a known source pedestrian. Exact copying provides the reference case; perturbing those copies tests redundant but slightly inconsistent observations.
It seems easy to just pre-process such cases away if they are undesirable. Why should we rely on the model for that?
UMOT's comparisons and MCTR's reported duplicate tracks show that redundant registration persists in evaluated tracking systems; TrackTrack explicitly adds a mechanism to suppress redundant track initialization. These examples show that preprocessing is already part of the solution, and that association of nearby, nonidentical tracks remains a substantive problem. Removing identical arrays alone does not resolve it. Our near-idempotence audit asks how a forecasting model responds when residual redundant records reach it. This makes downstream geometric stability a practical requirement alongside upstream duplicate suppression.
Tables 25–27 already include and m. With four added records, the new mean forecast displacements are:
| (m) | Social GAN | Social-STGCNN | AgentFormer |
|---|---|---|---|
| 0.02 | 0.0130 | 0.2180 | 0.1291 |
| 0.05 | 0.0301 | 0.2109 | 0.1331 |
These average all 20 forecasts and 12 prediction steps for original pedestrians, with equal averaging across the five ETH/UCY folds. Social GAN and Social-STGCNN use paired latent and standard-normal draws; AgentFormer recomputes its native deterministic proposals, paired by index.
The results establish sensitivity to near-duplicate histories, including with native proposal generation. The controlled noise scales are not estimates of real tracker-error frequencies; the tracking references establish why the redundant-record failure mode itself is practically relevant.
Thank you for the clarifications and pointing me to the relevant statements in Appendix D, which I had missed.
The reframing towards near-idempotence is exactly addressing the discontinuity question I had. I believe that this change in perspective and the expanded motivation in the response on geometric set data representation makes the motivation for studying this problem clearer and would be a positive change for the positioning of the paper.
In my view, the contribution is still somewhat incremental and the contributions don't fit together neatly yet:
The paper therefore provides both an a priori sufficient condition and a post-training assessment of existing implementations.
The key aspect that is missing for me is the connection from the a priori sufficient condition (which I find fairly weak by itself) and the experiments, mostly on models that have relatively little connection to it. I would have liked to see more impactful insights than the condition about max pooling, which is a known result as other reviewers have pointed out, and how that would inform model design.
In its current state, it does not meet the bar for acceptance for me, but I encourage the authors to study this interesting problem further to gain deeper insights first, which would significantly strengthen the paper in the future.
Response to Reviewer pvkZ (1/2): Architectural guarantees and contribution
Thank you for recognizing the relevance of multiplicity and continuity.
Proposition D.1 in Appendix D.2 gives an architectural sufficient condition: shared pointwise features, coordinatewise max pooling, and a decoder, F(A)=ρ(maxa∈Ah(a)). This construction is exactly duplicate-consistent. If the feature coordinates are Lh-Lipschitz and the decoder is Lρ-Lipschitz from the maximum norm, then dY(F(A),F(B))≤LρLhdH(A,B). These conditions can be assessed before training. They guarantee both exact consistency and a cardinality-independent bound for nearby supports.
The experiments evaluate trained models, but Proposition D.1 applies to an architectural class independently of the empirical audit. The paper therefore provides both an a priori sufficient condition and a post-training assessment of existing implementations.
This discussion is already present in §6 and Appendix D. D.2 proves the max-based Hausdorff bound above; D.6 connects it to the geometric audit; D.7 gives a hard-neighborhood discontinuity example; D.8 establishes conditional stability for an integral representation; and D.9 distinguishes structural guarantees from learned approximate consistency. We will bring the central result forward in the main text and ask that these existing sections be considered when assessing this point.
The additional Social-STGCNN analysis provides a concrete architectural diagnosis. Its off-diagonal graph rule assigns zero to identical observed displacement vectors and inverse distance to distinct vectors. With two nodes, self-loops of weight 1, and r=∥v1−v2∥, the normalized Laplacian is
\begin{pmatrix} 1 & -1 \\\\ -1 & 1 \end{pmatrix} \quad(r>0),\qquad L(0)=0.$$ Thus this graph representation is discontinuous at coincidence, independently of trained weights. The accompanying noise sweep measures forecast sensitivity at finite scales; it does not by itself prove a limiting jump in the trained predictor. > Given the inherently discontinuous objective here, achieving it seems tricky Could you clarify which objective you regard as "inherently discontinuous"? > The main conclusion seems to be essentially "you should measure to what extent models are sensitive to duplicates if you care about duplicates" Our conclusion concerns **geometric set data representation**: the datum is an underlying geometric object, and its observed points are finite approximations of that object. For a CAD shape, the surface exists independently of the chosen sample size or sampling density. A representation intended for that shape should therefore respect its geometry across different acquisitions. Exact idempotence is the zero-discrepancy case; near-idempotence addresses close, nonidentical observations. This exposes a bias in the benchmark protocol. Fixed-cardinality, uniformly sampled training and evaluation can reward models specialized to that acquisition law while leaving geometric consistency untested. High benchmark accuracy can therefore coexist with substantial representation changes when the same object is sampled differently. Our audits investigate this gap between learning the sampled registration and representing the geometric object. We will revise the title to "Beyond Permutation Invariance: Idempotence in Geometric Set Data Representation" and shift the main emphasis from exact idempotence to near-idempotence: stability under geometrically close registrations of the same object.