- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
FedHVLN lets navigation clients with incompatible model dimensions collaborate through a shared language interface and private projections. Its four stages combine human-instruction training, route-conditioned instruction generation, feature alignment, and pseudo-instruction navigation. Inference uses the human-instruction pathway. The paper's main contribution is applying this interface to federated VLN, where the meaning of an action depends on the client's current scene and viewpoint.
Critical strengths
-
Language is a sensible sharing interface (Sections 2 and 3.2). Navigation clients do not have a common set of action meanings, whereas they all need to interpret instructions. Sharing instruction features through private projections addresses this distinction directly and accommodates different native model widths.
-
The collaboration results give the interface a concrete purpose (Table 2). Cross-group aggregation improves SR by 3.07 points for wide clients and 3.25 points for compact clients while preserving each evaluated client's local training budget. Both groups benefit from experience contributed by models whose native parameters they cannot directly average.
-
The training machinery stays out of the inference path (Figure 2; Section 3.7). The Speaker and generator supply additional supervision during training, but deployment uses the human encoder and shared transformation. This is a useful property for integrating the method with an existing navigator. Table 1 also shows gains over FedVLN in homogeneous settings.
Critical weaknesses
-
The advance over shared auxiliary models needs a clearer statement (Section 2; Section 3.2). The paper identifies FedMRL as a close precedent, and the shared-module/private-projection design follows that general pattern. Conditioning a history-dependent navigator on language is a worthwhile application, but I am less clear about the new learning principle. The contribution would be easier to assess if the paper distinguished the VLN-specific insight from the underlying solution to incompatible parameter shapes.
-
The reason for the full training sequence is not yet persuasive (Sections 3.3–3.6). Stage 1 already learns a common instruction interface through local navigation. The Speaker provides a natural way to use routes without human descriptions, but it is less clear why those descriptions require a separate generator aligned to the human encoder. The paper describes the stages carefully; the missing connection is why this separation is especially useful for sharing navigation knowledge across clients.
-
The two language pathways depend on an important representation assumption (Section 3.5, Eqs. (3)–(4); Sections 3.6–3.7). Alignment initially brings pooled generator features toward the fixed human-encoder features. Stage 3 then adapts the generator locally and trains the shared transformation on its outputs, while inference applies that transformation to the human encoder. The method relies on compatibility surviving this adaptation. I would like a clearer explanation of how the local navigation objective is expected to preserve the relationship established by alignment.
Questions for authors
- In Sections 3.4–3.5, what is the intended advantage of a separate generator over encoding the Speaker's pseudo-instructions with the human-instruction encoder?
- In Section 3.6, how should the reader understand the balance between local generator adaptation and maintaining a useful shared transformation for the human-instruction path?
General Response
We thank all three reviewers for their careful reading of our paper. We summarize and clarify the following two themes: the gain of navigators of different widths from training together and the advantage FedHVLN retains in environments unseen during training.
Compact and wide navigators both benefit from training together. FedHVLN shares what clients learn about following instructions while keeping each client’s navigation policy and projection local. On R2R val_seen, we compared sharing across widths with sharing only among clients of the same width, holding each client’s training data and update budget fixed. Cross-width sharing increased success rate by 3.07 percentage points for wide navigators and 3.25 points for compact navigators. SPL also improved for both groups (Table 2 in the revised manuscript). Our replies to MKtb and FPxR explain how this result relates to broader architectural differences and when a navigator would need a different interface to use the shared instruction features.
In unseen environments, FedHVLN retains its advantage over both FedVLN baselines. FedHVLN led both baselines by at least 1.28 percentage points in SR and 0.81 points in SPL before pre-exploration, and by at least 0.60 and 0.65 points, respectively, afterward (Appendix A.6, Table 5). In response to your comments on generalization, we designed a new experiment to compare FedHVLN with both FedVLN baselines before and after pre-exploration. We used the same compact-to-wide navigator ratio in the 11 R2R
val_unseenenvironments as in the seen environments and reserved all 2,349 human-written instructions for evaluation. Before pre-exploration, FedHVLN achieved 53.77% SR and 49.52% SPL, compared with 51.13% and 46.50% for wide FedVLN and 52.49% and 48.71% for compact FedVLN. After 30 rounds of pre-exploration on local routes with Speaker-generated instructions, FedHVLN reached 67.18% SR and 64.47% SPL, compared with 66.58% and 63.82% for wide FedVLN and 66.20% and 63.27% for compact FedVLN (Table 3 in the revised manuscript). Mixed-width FedHVLN outperformed both single-width baselines on both metrics before and after pre-exploration.In our individual replies, we also report what training on additional routes contributed under a matched update budget, how the adapted model uses human instructions at inference, and the communication and computation costs of the four training stages.