- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper develops certificates for inverse learning from incomplete trajectories in finite deterministic absorbing MDPs. Its main result bounds the loss incurred by retaining a cached policy within a two-terminal module when both completed counts and incoming throughput change. The guarantee accommodates cyclic modules and inexact caches, and a potential adjustment can tighten the certificate without changing the cached policy. Supporting results certify improvements in the record score and translate a frozen fitting gap into guarantees on complete-path distributions. The experiments investigate certificate sharpness, acceptance cost, and cache retention.
Critical strengths
-
The central certificate can be evaluated using the cached flow and the updated problem, without solving for the updated optimum. It also controls the KL divergence between complete-path distributions in both directions. Its coverage of cyclic modules and inexact caches broadens the scope of the result.
-
The mean-exposure bound is an appealing result. The repeated-loop construction in Appendix H demonstrates sharpness even when path lengths are unbounded. The flow-gap and Bregman decompositions also help clarify where the guarantees come from.
-
The paper distinguishes the behavioral policy from the completion distribution induced by the regularized score, and separately treats block accuracy, record-score improvement, and module reuse. These distinctions help prevent overinterpretation of the certificates.
Critical weaknesses
-
The practical benefit of reuse remains unclear. In the module study, a single refinement step retains only seven additional caches after actual problem changes, while increasing median batch time by approximately 24% relative to warm-started full solving. These results leave the computational value of the proposed reuse mechanism uncertain.
-
The method requires positive feasible flows and two-terminal modules without internal origins. The small real-data pilot does not establish how much of a realistic network satisfies these conditions or how much computational work falls within eligible modules.
-
The reported numerical certificates apply to stored record kernels. Kernel construction, flow feasibility, and Q reconstruction are checked through numerical audits, leaving a gap between certification of the stored-kernel problem and certification of the complete learning pipeline.
-
The results concern a particular, generally unnormalized score. Because alternative score conventions can produce different completion weights, predictive evaluation would help establish why this choice is appropriate for incomplete demonstrations.
Questions for authors
-
Could the module results be broken down by whether completed counts, incoming throughput, or both actually change, and by whether the cache is exact or inexact? A timing breakdown separating refinement cost from the refitting work avoided would also help clarify the practical benefit.
-
In the real-data pilot, what proportion of the eligible flow or fitting cost is covered by the retained modules? How much support is excluded by the acyclic restriction?
-
Could the paper provide a worked numerical error budget for one accepted stored-kernel update, showing the bounds on kernel-construction and reconstruction errors that would be needed to extend the certificate to the ideal objective?
Awaiting Your Feedback on Our Rebuttal
Dear Reviewer yCXm,
Thank you again for your review. As the discussion period is drawing to a close, we would appreciate it if you could take a look at our rebuttal. We would be happy to clarify any points or address any remaining concerns while the discussion is still open.
Best regards,
The Authors