- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a novel auxiliary objective for the Pathfinder long-range same-path task, derived from relative homology. The task boils down to identify if two endpoints of dashed curves belong to the same curve or not. The authors’ proposed target is , where is the convex hull of , i.e. the line segment that connects both endpoints. This target equals the number of components the line intersects minus 1. The authors predict this number during training in a supervised manner and discard the corresponding head during inference. The main result is presented in Table 1, in which it is shown that a CNN, trained with this auxiliary target, reaches test accuracy versus with the label-only training. Authors also ablate other auxiliary targets: predicting alone and permuting across the training dataset. These auxiliary targets don’t improve the performance, therefore showing that the additional head and the loss term is not enough. Only the correctly paired relative target succeeds, and the authors read this as evidence that the query-anchored topology, and not the extra supervision by itself, drives the gain.
Critical strengths
An observed result is crisp and real. Changing the training loss by adding the proposed auxiliary target allows a plain CNN to perform extremely well on the given task. Using the released code, I reproduced the results of Table 1 for seed 23 and obtained accuracy with the proposed loss, while obtaining only for the label-only task.
The proposed solution is cheap.: The target for the auxiliary task can be computed from the input and is used only during training, therefore adding nothing to the inference cost.
Critical weaknesses
W1. The topological machinery buys nothing. Authors put relative topology in the title, devote the first figure in text, and spend the whole section explaining the topological construction. However, the gains from the topological machinery are marginal: Figure 1 and the half of the section explain the motivating example, but not the implemented method, the method itself boils down to the calculation of the number of connected components intersected with the interval; the long exact sequence calculation in the Appendix A also gives effectively nothing.
W2. Is it the relative homology that helps? It is unclear if it is exactly the relative -th homology that helps the method to excel. Seems clear that the subtraction of can be dropped (because it can be absorbed by the output bias), and the task can be reduced only to the prediction of the number of intersections. And, given that the interval between the endpoints can pass the curve between two dashes, or, if the curve is twisted enough, the interval might pass it twice, the need of the concrete value is also questionable. The authors seem to ablate this by randomly permuting the auxiliary target of the training set, showing that the test accuracy stays on the level of the label-only task, but it doesn’t automatically prove that the concrete targets must be exactly .
W3. Is it topology that matters? An ablation I would love to see is, instead of using the proposed auxiliary loss, take a simple one, that would also be related to the query, e.g. the length of the line segment. It would allow one to decouple the potential topological explanation from the “query localisation”. Maybe all you need is just any target related to the query itself? My preliminary checks (seed 23) support this: I tested two possible auxiliary targets, the number of dashes in the intersection of a line segment between endpoints, and the distance between them. In both cases, the test accuracy was substantially higher, than for the label-only model ( and at the 200th epoch respectively). I believe such comparisons are essential for the experimental analysis, and I would love to see that kind of analysis in the paper.
W4. The auxiliary loss is not ablated. The motivation behind the auxiliary task is understood, the concrete implementation as the Huber loss, however, is not justified. Why do the authors prefer the Huber loss over MSE, L1, or a classification (since the target is just an integer in )? Authors don’t state the threshold , used in the Huber loss (the code use ), authors don’t ablate , using for every Pathfinder run, and using for DRIVE. I would love to see the sweep over in order to validate if the observed jump is robust or not.
W5. The paper needs a rewrite. The paper reads unclear sometimes, some parts of the texts are too repetitive, others - too vague. For example, authors compare the previous related work with their proposed approach in every paragraph of the corresponding section, before their approach is even described, making it hard to follow. The section of related work requires special proofreading: e.g. authors mention “Towards a general purpose CNN for long range dependencies in D” by Romero et. al., saying that, together with other papers, this paper receives 1-dimensional pixel sequences. However that paper has 2D experiments as well (see, for example, Figure 3).
Questions for authors
Does paper really require topological treatment? The topology-inspired auxiliary target? The exact implementation of the loss as proposed? If the answer to any of the questions is “yes”, please explain why.
5 comments
Response to Reviewer BJBR: additional ablation on the auxiliary weight
Follow-up to W4 (auxiliary weight). As for the weight ablation, we tested on the standard Pathfinder setting (plain stack, 4,000 training scenes). Changing only the auxiliary weight from to , the value used for DRIVE, gives 98.78 ± 0.22% validation-selected test accuracy (98.15 ± 0.39% at ; +0.63 points; mean ± sample SD over three seeds), and all three seeds exceed 90% validation accuracy. Although it is not complete ablation study, this supports robustness to the choice of the auxiliary weight.
I thank the authors for the detailed response and the additional experiments. The experiments provide new information, so I will raise my Informativeness score from 1 to 2.
However, I’m not really convinced by the presented topological argument. The topological formulation as “a principle for constructing supervision” seems to me a weak justification for invoking the machinery of algebraic topology. In the plane, with a contractible , the pair has non-trivial homology only in degrees 0 and 1, so this “principle” boils down to counting either the components the segment meets or the loops it closes, and which of the two is used depends on the dataset and its labels (see Appendix H, “Choice of homological degree”).
As for W3, the target I suggested was not really tested. I asked for local intersections: the number of times the segment crosses ink, with repeated crossings of the same curve counted separately, i.e. the number of ink pieces along the segment, not the number of ink pixels near it. This pixel count lies somewhat in-between my target and the segment length, which may be why it turns out barely more informative than the length at 2,000 scenes. This 2,000-scene setting is the only evidence that the topological target beats non-topological ones: besides the 4,000- and 2,000-scene settings, the authors also report dense clutter (16,000 training scenes), and there, with identical scaling, the pixel count again matches (Table 10). Unlike the pixel count, the piece count is an integer on the same scale as , so it can be compared without rescaling, like the unscaled reference. The target I originally proposed might well match at 2,000 scenes too. I would appreciate it if the authors added this experiment to Table 10.
Additionally, the segment-length control on Satellite PathX seems to be uninformative by design, since the query separation is restricted to px (§5), and Appendix H acknowledges that “the segment-length target varies little because separation is restricted by the dataset design”. Predicting a near-constant carries no information, so its accuracy being close to label-only is expected, and it cannot show that “this non-topological query-dependent target does not provide the benefit”. The other controls are a scene-only count and a permuted target, so there is no informative non-topological control on this task at all.
More generally, for a method that is, in my view, weakly motivated, the evidence has to come from extensive ablations, and these are few. The loss, the weight and the Huber threshold are swept only for (Appendix E.2), while the alternative targets in Table 10 are run in a single configuration. It might be that, with tuned hyperparameters, the pixel count would match . Given all this, I keep my rating.
Response to Reviewer BJBR
Thank you for independently reproducing the gain and testing alternative targets.
W1–W2 (role of topology). We agree that your component count and our proposed relative target coincide up to a constant: Δβ0=k−1, where k counts the foreground components met by the connector. Although this elementary count can be computed without homology, our formulation provides a principle for constructing supervision. The construction also gives conditions for invariance under data augmentation (§3.2). The same principle also generates the area-filtered degree-one road target (general response; SI §H).
W3 (simpler query-dependent targets). We tested both suggestions. Segment length is endpoint distance; we interpret the intersected-dash count as the number of foreground pixels within 1.5 pixels of the queried segment.
Validation-selected test accuracy (%), mean ± sample SD over three seeds; each target is divided by its training-set SD (this makes the new labels more stable for learning). Bold marks the highest mean, not significance. Budgets (epochs) approximately match gradient updates (SI §E.1).
Both alternatives help at 4,000 scenes as you expected. However, with a smaller data size at 2,000 scenes, the proposed relative target retains its benefit. Thus, we speculate that the topology also plays a role in addition to query localisation.
W4 (loss and weight). The exact Huber formulation is not required: L1 and integer cross-entropy also succeed. SI §E.2 specifies the default δ=1 and reports the loss, δ and λ sweeps; MSE and excessive auxiliary weight are less reliable.
W5 (presentation). We rewrote §2, explicitly separating two-dimensional continuous-kernel CNN experiments from one-dimensional sequence models.