- Rating
- 3
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduced the JAXOLOTL a benchmarking tool can be use in multitask RL where tasks expressed in LTL. They have implemented their tool using JAX, and included four multitask environments. Finally they have shown their efficiency of their tool comparing with the original implementations and also perform a benchmarking on the popular RL-LTL methods.
Critical strengths
1- The motivation behind the reason why we need a unified tool for RL-LTL is presented well.
2- The engineering side of the project is valuable. Using JAX gives JAXOLOTL an significant advantage over original implementation.
3- Experiments are thorough and well prepared. I like the stress test had been done in sec.5.2.2 and 5.2.3. The observations from these two experiments are valuable for RL-LTL community who want to develop their own method. Now they know what are the gaps and how they can fill this gap.
Critical weaknesses
I thank the authors for their hard work implementing JAXOLOTL, however there are some issues which I split them in two main categories: Formalism and Writing, and Implementation.
Formalism and Writing
1- Paper has few formalism errors such as: should be , or you need to add the constraints that the sum of probabilities are not going to be bigger than 1. In addition, is defined as , but in line 159 eq, it conditioned on the specification . Further, in eq (line 159), is sampled from the on the right side, how we have on the left side as well. Are we looking for a fixed specification? Also should be uniform distribution, since if it is an arbitrary distribution, the policy can get biased to optimized satisfying only some of specifications.
2- In my opinion the most important thing that a new benchmark should have, is how easy it can be used for new methods. The Modular Abstractions section, could not deliver that, I think more explanation, demo or part of a code could help to show this.
Implementation
3- Since we call JAXOLOTL as an unified benchmark it should have the environments to distinguish different methods with different abilities, for example in PlatoLTL (Cloete et al., 2026) they introduced new environments to show their advantage over parametrized settings, so I think JAXOLOTL should include these environments as well.
4- In Fig.5, we see that most (3/4) of settings have already been dominated by some of the methods, and there is absolutely no room for improvement for new methods. So new methods can not show their superiority over the RL-LTL baselines if they use this benchmark.
Questions for authors
My main questions are (1) what are the features proposed for the researchers want to work on RL-LTL field and develop their own method, (2) is there a room for improvement so they can work on, (3) How diverse are the environments in which different methods demonstrate their respective advantages?
5 comments
Rebuttal Part 2
How diverse are the environments in which different methods demonstrate their respective advantages?
The suite exposes complementary strengths: GenZ-LTL with observation reduction handles large proposition vocabularies well, whereas StructLTL and DeepLTL perform better on the controlled ZoneEnv-NM test requiring non-myopic reasoning (92.7% and 83.6%, versus 70.0% for GenZ-LTL). StructLTL achieves the highest finite-horizon success rates on both Warehouse variants, which permit simultaneous propositions (Figure 6 and Table 1). For detailed descriptions of the differences between environments, see Appendix E.
We hope that the new results and discussion above are helpful and have clarified some of the points. Please let us know if you have any other questions or remarks!
Thanks for your detailed rebuttal.
formalism
Yes, I agree most of the issues are resolved but I the inconsistency regarding the policy is still there, defined as a mapping that only gets as the input, then in ln159 same notation conditioned on the LTL spec. Either authors should change this notation chnage to something else or revise the current one. Also can you please specify a meaningful task that we need some distribution which consequently lead to some bias?
In my opinion the most ...
Resolved.
New challenging experiments
In the process of making the experiments more challenging, which feature of the envs targeted so now we have harder tasks? More feature, propositions, formulas ? This insights could be useful if added to paper so researchers can target to handle those aspects.
PlatoLTL
Unfortunately I am not convinced. In PlatoLTL they have all LTL-RL baselines, if JAXOLOTL is unified it should contain both PlatoLTL and it's envs.
Diversity
I think PlatoLTL and it's experiments is a good response to this concern. In my humble opinion adding that would strengthen the paper more than what it is right now.
Many thanks for your review. We are happy to read your positive assessment, especially regarding the “valuable” engineering and “thorough and well prepared” experiments. Please see below for answers and comments to the points raised. Please note that we have updated the paper with new experiments and further discussion.
Many thanks for pointing these out. We fixed these errors in the updated version of the paper: we corrected the transition kernel, introduced LTL-conditioned policies before the objective, and removed the specification from the left-hand side. We retained the wording of “arbitrary distribution” since we make no assumptions on uniformity (in fact, different methods employ different task distributions, e.g. via curriculum learning), but we’re happy to discuss this further.
We agree with this and think this is a valuable addition to the paper. We have hence added an overview of Jaxolotl’s code architecture in the new Appendix C, together with a brief description of how to extend the framework with new methods. As an example, we explain how StructLTL is implemented by reusing relevant components from DeepLTL. We will furthermore add a detailed developer guide to the final repository, with more detailed explanations and references to the code.
This is a good point, which was also raised by reviewer RyrC. We hence updated the paper with a new challenge track of substantially harder environments. This includes QuadrotorEnv as a new environment, which is an adaptation of ZoneEnv with a 3D quadrotor instead of the point agent. The policy directly controls the thrusts of the four motors, forming a much harder nonlinear control challenge. The challenge track furthermore consists of a scaled up version of ZoneEnv with 8 propositions and non-myopic dynamics, the Franka environment with 12 propositions, and Warehouse-hard with six distinct objects, leading to 256 total valid proposition assignments.
We observe that existing methods struggle in these challenging domains: while GenZ-LTL still performs well in QuadrotorEnv and FrankaZoneEnv-12, its performance severely degrades without environment-specific processing. It also leaves significant headroom in ZoneEnv-NM-hard and does not apply to the hard Warehouse variant. The other methods similarly struggle, achieving maximum success rates of 79% (QuadrotorEnv), 69% (ZoneEnv-NM-hard), 29% (FrankaZoneEnv-12), and 31% (Warehouse-hard) on finite-horizon tasks.
We hope this challenge suite will serve as a useful testbed for future research, demonstrating the need for both non-myopic reasoning and scaling with atomic propositions.
PlatoLTL’s parameterised environments evaluate an important complementary capability: generalisation to previously unseen (parameterised) propositions. Our current benchmark focuses on compositional generalisation over fixed, finite proposition vocabularies, enabling controlled comparisons among the six included methods. Evaluating parameterised propositions would require adapting these methods and introducing a corresponding evaluation protocol. We therefore leave this extension to future work. The revised suite nevertheless distinguishes existing methods along several complementary dimensions, including non-myopic reasoning, proposition scaling, and simultaneous proposition satisfaction.
Researchers can reuse the training and evaluation loops, curriculum machinery, observation encoders, and LTL-processing infrastructure while implementing method-specific task representations, wrappers, policy models, and evaluation agents. Appendix C describes these extension points and their configuration, and illustrates how StructLTL reuses DeepLTL’s training/evaluation wrappers and evaluation agent while introducing a new task representation and encoder.
Regarding future research directions, we discuss this in the revised Section 5.3 and Conclusion (Section 6). Specifically, what is currently missing is a method that has all of (i) non-myopic reasoning, (ii) favourable scaling to large proposition vocabularies, and (iii) applicability without environment-specific preprocessing. Interesting potential directions include learning observation reduction functions via feature selection or attention mechanisms, and relaxing the assumptions of known propositions and accurate labelling functions.