- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Enterprise-JEPA is an action-conditioned latent world model for text-based tool-using agents in enterprise environments. Where existing LLM-based world models autorgressively generate the imageined next tool observation, Enterprise-JEPA predicts that observation's representation in a single forward pass, and decodes the predictor's final hidden state into seven interpretatble categorical fileds describing the action's operational consequence: action type, execution staus, error signature, progress signal, side-effect type, inforation sufficiency, and task completion. The model have a Qwen3-embedding-0.6B encoder feeds a 1024 linear projector with LayerNorm then a Transformer predictor over a nine-token sequence: eight recent tool-output representations plus one context token. Actions are not separate tokens. Stage 1 learns latent dynamics self-supervised, aligning the predicted zt; stage 2 freezes encoder, projector and predictor and fits the seven heads with calss balanced cross entropy.
Critical strengths
- it is the first action-conditioned JEPA model for text-based llm tool agents in enterprise settings, predicting tool0induced transitinos in latent space rather than reconstructing future observations.
- The stage 1 ablation is internal consistent and makes it convincing, The default skeptical hypotehesis for any two stage paper is that the labelled stage 2 does not work and the representation learing is decoration. They have tested that the collapse to 0.404 falls below both the small corpus model and the LLM -WM baseline. More telling than the magnitude is the pattern, it degrades hardest on execution status and error signature.
- The paper also ran planning control that most planning paper skip, they have shown that the gains are from the world model rather than from candidate generation plus reflection.
- The annoation schema was pruned by measured reliability rather than intuition. and Appendix C is better than most human annotation sections, let alone LLM - annotation ones:two passes, three strong raters plus a separte tie break rater.
Critical weaknesses
-
The headline claim is broader than Table 3 supports, and the one regression is never metioned. The abstract's beam-seach sentence is narrowly accurate, but the paper proposes three harnesses, and Enterprise-JEPA with ITP-I on automationbench scores 0.136 against 0.174, there is a drop.
-
The hand engineered planner may be doingmuch of the work. Appendix D deines the ranking function with roughly fifteen free constants: five field weights, six category utilities, seven lambdas, read pressire K-2 , veto threshold at 0.6 and 0.5 and an acceptance margin. No tuning protocol is stated, no help out split for selectionand no senstivitiy analysis beyond the candiate count and horizon.
-
there is no decisive baseline; the no stage 1 leaves the projector and predictor at random initilisation, which makes it a strawman for question the paper's framing actually raises. The needed control is the same seven heads, same architecture, with the action representatino available, trained on the same labels but without the stage 1 alignment objective.
Questions for authors
-
May need to explain the table 3 0.136 vs 0.174 better, the 3.8pp drop and the largest regressino in the table.
-
Table 4, both controls fall below the no WM baseline(0.342 and 0.338 and 0.371). which implies the beam-search scaffold is net harmful without informative predictions. Could you state the decomposition explicitly: how much of the 0.404 is recovering harness overhead, and how much is net new value?
-
how were the roughly fifteen constants in appendix D is selected, and on what data, the five field weights, the category utilities, and etc. are they tuned on enterprise ops gym? how much of the 0.404 holds on a benchmark they were not tuned on?
3 comments
Author Response to Reviewer du5b (Part 2)
Item 3: Q2, Decomposition of the 0.404 success rate
Thank you for raising this critical point. Relative to the no-feedback control (0.342), informative predictions add 6.2 points: 2.9 recover the baseline level (0.342 → 0.371) and 3.3 are a net gain over it (0.371 → 0.404); relative to shuffled predictions, the split is 3.3 + 3.3. To decompose the contribution of the predictions further, we added a ranking-only control.
We added two controls on EnterpriseOps-Gym (80 tasks, , , three runs per condition). The actions-only control retains JEPA-based candidate ranking and gating but shows the policy only the selected actions, withholding the predicted states. The unranked actions-only control assigns equal scores to all candidates and selects the first generated candidate; the policy again receives only the selected actions when the plan passes the confidence gate. Both controls retain the full planner's candidate generation, scoring rules, vetoes, confidence-gate rules, and policy-call budget. These comparisons distinguish the contribution of prediction-guided selection and gating from that of explicitly displaying predicted states. Table R combines these controls with the published results.
| Condition | Candidate ranking | Shown to the policy | Success (%) |
|---|---|---|---|
| No world model | — | — | 37.1 ± 0.6 |
| State-output LLM-WM | LLM-WM predictions | actions + predicted states | 37.1 ± 1.9 |
| JEPA, unranked actions only | none | selected actions only | 36.2 ± 2.2 |
| JEPA, actions only | JEPA predictions | selected actions only | 39.6 ± 0.7 |
| Enterprise-JEPA | JEPA predictions | actions + predicted states | 40.4 ± 0.7 |
Table R. Candidate selection versus explicit predicted-state advice. Success is reported as mean ± standard deviation across three runs.
The actions-only control retains most of the full planner's performance: 39.6% versus 40.4% success rate. It exceeds the unranked actions-only control by 3.4%, suggesting that JEPA-guided plan selection and gating contribute to the improvement beyond supplying candidate actions and additional deliberation alone. Adding the predicted states and advice provides a further mean improvement of 0.8%. These results support the value of using learned world-model predictions for plan selection. We include this analysis in Appendix F.9 and the discussion of Table 4.
Item 4: W3, Architecture-matched control without the alignment objective
Thank you for proposing this valuable experiment.; we ran exactly this control. We added this baseline (Architecture-matched baseline). It starts from the same weights as the "no Stage 1" row, keeps the architecture, AdaLN action conditioning, and heads, and trains the same parameters as Stage 1 (top four encoder layers, projector, predictor) together with the heads on the 17,066 labeled transitions, with the Stage-2 data, loss, and schedule but without the latent-prediction objective. We also fine-tuned the Stage-1 checkpoint end to end on the labels (JEPA init + sup. FT).
We report the mean ± standard deviation across 5 seeds below and will include the results in the revised version of the paper along with Table 2.
| Model (5 seeds) | Avg. macro-F1 | Avg. macro-recall | Exec. status F1 | Seeds exec. ≥ 0.6 |
|---|---|---|---|---|
| LLM-WM, expanded | 0.557 ± 0.003 | 0.556 ± 0.003 | 0.387 ± 0.003 | 0/5 |
| JEPA, no Stage 1 | 0.403 ± 0.002 | 0.417 ± 0.002 | 0.264 ± 0.005 | 0/5 |
| JEPA, small corpus | 0.578 ± 0.006 | 0.606 ± 0.001 | 0.583 ± 0.046 | 4/5 |
| Architecture-matched baseline (new) | 0.587 ± 0.013 | 0.608 ± 0.012 | 0.458 ± 0.093 | 1/5 |
| JEPA init + sup. FT (new) | 0.567 ± 0.053 | 0.585 ± 0.063 | 0.475 ± 0.126 | 2/5 |
| JEPA, expanded | 0.602 ± 0.003 | 0.623 ± 0.002 | 0.611 ± 0.005 | 5/5 |
Table S. Next-state prediction, mean ± std over five seeds.
- Enterprise-JEPA with the expanded corpus is the best and the most stable model. It achieves the highest average macro-F1 (0.602 ± 0.003) and average macro-recall (0.623 ± 0.002).
- Stage 1 makes execution-status prediction reliable: Enterprise-JEPA achieves a mean F1 of 0.611 and exceeds 0.60 in all five seeds, whereas the architecture-matched baseline reaches this threshold in only one of five seeds (mean 0.458 ± 0.093), with a nearly 20× larger standard deviation. Predicting whether an action succeeds is exactly what next-observation prediction learns.
These results show that the latent-prediction objective, combined with unlabeled data, improves both execution-status prediction and seed stability.
Official Comment by Authors
Thank you for the time and effort you have devoted to reviewing our submission. We have responded to your comments and revised the manuscript to address the concerns you raised. Our responses to the other reviewers may also provide useful clarification on related points.
As the discussion period draws to a close, we would welcome any further thoughts or remaining concerns and would be happy to address them.
Thank you again for your thoughtful consideration.
Author Response to Reviewer du5b (Part 1)
We thank Reviewer du5b for the encouraging assessment of our work as the first action-conditioned JEPA model for text-based tool-use agents. We also appreciate their insightful comments on our ablation choices (including Stage 1 and the planning controls), as well as the thorough description of the annotation protocol. We address your concerns and questions below.
Thank you for pointing out this omission. We agree that our headline claim should distinguish performance across planning harnesses. On AutomationBench, Enterprise-JEPA with ITP-I scores 0.136, below the no-WM baseline of 0.174, a drop of 3.8 percentage points. We will explicitly report this regression and revise the claim to state that Enterprise-JEPA improves over the baseline on all five benchmarks under beam search, rather than across all harnesses.
Our analysis of the AutomationBench logs suggests that the outcome depends on the interaction between the world model, harness, and benchmark:
Performance varies across harnesses. The same Enterprise-JEPA checkpoint scores 0.180 with Revision and 0.192 with beam search, compared with the 0.174 no-WM baseline. Of the 26 tasks lost under ITP-I, 19 are solved by Enterprise-JEPA under Revision.
ITP-I is challenging for all three evaluated world models on AutomationBench. It is the lowest-scoring harness for Enterprise-JEPA, the state-output LLM-WM (0.167), and Qwen-AgentWorld (0.151). This pattern suggests a harness-related effect, although it does not rule out a contribution from the world model.
The logs reveal problematic rollout and execution behavior. ITP-I asks the policy to generate a four-step hypothetical rollout and returns it as free-text foresight. On AutomationBench, 11% of these rollouts end early with an unparseable action, and 18% repeat the same action at every imagined step. During execution, repeated tool calls account for 14% of calls versus 3.3% for the baseline, and 21.7% of tasks reach the 50-turn limit versus 8.7% for the baseline. Some lost tasks also complete with partial scores because of incorrect arguments or omitted writes.
These observations are consistent with ITP-I being less effective on this benchmark, but they do not establish a single cause for the regression. Beam search uses predictions differently, to rank and veto candidate plans, and achieves 0.192 with Enterprise-JEPA on the same benchmark. We will add the regression, the log analysis, and a per-task breakdown to the revision, and qualify the paper's claims accordingly.
We thank the reviewer for highlighting the need for further clarification.
Design principles. Execution status (1.0) and progress signal (1.2) receive the largest weights because they directly indicate whether an action succeeds and advances the task. Complementary fields receive smaller weights: side-effect type (0.6), information sufficiency (0.5), and error signature (0.4). Category utilities use a common scale from −1 to +1, representing unfavorable and favorable outcomes, respectively. Neutral progress receives a utility of −0.3 to favor actions that advance the task, while intended state changes receive neutral utility. Read-pressure and required-operation coverage terms balance information gathering with the state-changing operations needed to complete the task. We will add these design principles and the rationale for the numerical values to Appendix D.