- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper discusses that recurrent memory for partially observed control should depend on what the observation mask hides. Hidden velocities define an observable “derivative regime,” where memory combined with the current observation can recover the hidden state. Hidden positions define an unobservable “integral regime,” where a plain GRU suffices. DuoCell splits a fixed state budget between a GRU and a structured prior–innovation recurrence, with a gated reread of the current observation. On POBAX masked MuJoCo, it outperforms twelve baselines in the derivative regime, including a width-matched GRU (+0.52 normalized IQM) and a tuned TransformerXL. A preregistered test on four held-out morphologies confirms the predicted direction of benefit in both regimes. However, a GRU with the same reread shows the same pattern and outperforms DuoCell out of domain.
Critical strengths
-
Predictive insight beyond the benchmark split. Building on POBAX’s masks, the paper predicts when observation rereading should help and confirms the predicted direction at the regime level on held-out tasks. This offers useful guidance for memory design beyond DuoCell itself.
-
Strong, carefully controlled empirical results. DuoCell beats twelve baselines in the derivative regime, including a width-matched GRU and a tuned TransformerXL. Capacity controls and factorial ablations strengthen the evidence for its sample-efficiency gains.
Critical weaknesses
-
The evidence supports the criterion, not DuoCell. On the eight held-out environments, a GRU with the same gated reread beats DuoCell in both the derivative and integral regimes (Table 6), despite trailing it in domain. What generalizes is the reread, not the heterogeneous cell.
-
The evaluation is too narrow. Training runs for 1M steps, versus 50M in POBAX. At 5M steps an untuned TransformerXL closes most of the gap, and at equal wall-clock the two tie. DuoCell's advantage may not hold at the benchmark's own training scale.
Questions for authors
-
Mixed-regime evaluation. DuoCell is motivated by platforms that mix both regimes, but every experiment hides either all velocities or all positions. Can the authors test a mixed mask?
-
Iterated correction. The structured path corrects once per step. Would iterating the correction–reread loop within a step help, as in an iterated Kalman update? Could a single correction step explain why zeroing the innovation has no detected effect?