acceptodds
Under review as a conference paper at ICLR 2027

FIRE: Failure-Informed REcovery Post-Training for Real-Time Visual Combat

Abstract

Real-time visual combat exposes a mismatch between how human demonstrations are collected and where a learned policy needs supervision. Expert data are typically collected open loop and emphasize successful trajectories, leaving policy-induced failure states underrepresented. As the policy evolves, these deficiencies also shift, making additional generic demonstrations increasingly inefficient. We argue that post-training should instead close the loop between policy experience and human data acquisition, using rollout failures to determine what should be demonstrated next. We introduce FIRE: Failure-Informed REcovery Post-Training, a closed-loop framework that turns the policy's evolving failures into a curriculum for human supervision. Rather than collecting more generic expert play, FIRE uses recurring rollout failures to identify missing recovery behaviors, acquires demonstrations targeted to these deficiencies, and feeds them back into subsequent policy updates. A multimodal coach translates recurring failures into actionable recovery requests. The resulting demonstrations are organized in a recovery library and retrieved according to the encountered failure modes during subsequent supervised updates, while rollout feedback supports critic-free policy optimization and selective replay. Across seven Black Myth: Wukong bosses and two character configurations, FIRE improves normalized Boss HP Reduction over supervised fine-tuning (SFT) on all 14 boss-configuration pairs and Win Rate on 13. Under an equal additional demonstration-video budget, FIRE achieves higher final HP Reduction on all three controlled encounters than either additional successful demonstrations or Static recovery targeting only the initial SFT policy's failures. Supplementary videos are available at https://fire-anonymous-supplement.pages.dev/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
visual game agents, human-in-the-loop learning, learning from demonstrations, policy post-training, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hDyEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FIRE, a post training framework for real-time visual combat. The VLA actor receives raw pixels as input and outputs keyboard/mouse actions. FIRE starts from an SFT policy trained on expert play and runs 10 online rounds with 2 streams (RL stream & Supervised stream). The key component is failure-informed recovery demo acquisition. A VLM diagnoses mistakes from rollouts and groups them into failure modes. Recurring failure modes make requests for record expert’s recovery demonstrations, and retrieved by failure label for policy updates. FIRE reports improvements in Boss HP Reduction and Win Rate across seven in-domain environments under two attribute settings compared with the considered baselines. The paper also provides several ablation studies to analyze the contributions of individual components

Critical strengths

S1. The paper uses a VLM to automates the decision of what human experts should record (failure diagnosis, recording requests, and demo labeling and retrieval). Human experts only record offline demos between rounds, without required continuous monitoring, takeover, or state resets. This matters in real-time game, where such supervision is hard to provide.

Critical weaknesses

W1. The proposed framework largely combines existing components, such as aggregating human demonstrations, VLM-based failure detection, and VLM-derived rewards. While the integration is practically useful, the methodological novelty appears limited.

W2. The main results in Table 1 primarily compare variants of the proposed method and therefore resemble an ablation study rather than a comprehensive comparison against external baselines. The evaluation would be substantially stronger with additional baselines, such as standard RL methods (e.g., PPO) and intervention-based post-training methods. Without such comparisons, it is difficult to assess how much of the reported improvement is attributable specifically to FIRE relative to existing alternatives.

W3. All experiments appear to be conducted with a single random seed, and the paper does not report standard deviations or confidence intervals. Given the potentially high variance in RL training and evaluation, results across multiple seeds are important for assessing the reliability and statistical significance of the reported improvements. Together, W2 and W3 make it difficult to draw strong conclusions from the empirical results.

Questions for authors

Q1. Could you add more external baselines, such as standard RL and intervention-based post-training methods? these methods should use the same SFT policy initialization and a comparable training/data budget to ensure a fair comparison.

Q2. Could you report results across multiple training seed (3+) and confidence intervals on experiments?

5 comments
AuthorsOpenReview ↗

Response to Reviewer hDyE(Part 1)

Thank you for your careful review. We have added the new experiments and analyses to Appendix B of the revised PDF, with the additions marked in green.

W1. What FIRE contributes beyond its individual components

While the integration is practically useful, the methodological novelty appears limited.

FIRE closes the loop between policy failures, human demonstration collection, and policy updates. Its contribution is an adaptive correction mechanism that connects curated replay, VLM diagnosis, and expert demonstrations through shared policy experience and an evolving failure vocabulary.

Rule-based feedback and VLM assessments jointly identify informative successes and mistakes for replay, allowing subsequent RL updates to revisit them. The VLM also diagnoses mistake chunks and turns them into specific recovery requests. FIRE prioritizes uncovered failure modes, then allocates recording effort by failure frequency relative to demonstration coverage. Human experts recreate the relevant situations and demonstrate recovery actions. The VLM assigns these demonstrations the same failure labels used for policy mistakes, enabling retrieval by boss and failure mode for supervised updates alongside RL. Replay revisits the actor’s experience, while experts supply corrective action examples. VLM review and human recording occur outside real-time gameplay.

The evolving labels connect diagnosis, collection, and training. During collection rounds, newly observed failures can introduce more specific labels and redirect recording and retrieval as the policy changes. The matched-video controls retain hybrid feedback, replay, RL, and supervised learning while varying demonstration acquisition. Our added fixed-label control further retains VLM diagnosis and adaptive recording priorities but keeps the initial failure vocabulary fixed. We evaluate Round 10 with 20 episodes per condition.

Wind and Bear denote Black Wind King and Black Bear Guai. Cells report HP reduction / win rate (%).

MethodWind (50/600)Bear (50/600)Wind (native)Bear (native)
Static recovery63.27 / 2568.88 / 3054.46 / 553.50 / 10
Fixed labels + priority64.19 / 2570.43 / 3054.31 / 056.86 / 10
FIRE70.95 / 3575.07 / 4058.57 / 1059.76 / 20

FIRE exceeds the fixed-label control by 2.90–6.76 points in HP reduction and 10 points in win rate across all four conditions. Priority alone improves HP reduction over Static recovery in three conditions, without improving win rate. These results support expanding the failure vocabulary alongside adaptive recording priorities (Appendix B.3, Table 13, p. 32).

AuthorsOpenReview ↗

Response to Reviewer hDyE(Part 2)

W2. Comparison with external baselines

The main results in Table 1 primarily compare variants of the proposed method and therefore resemble an ablation study rather than a comprehensive comparison against external baselines.

We added PPO and HG-DAgger, and FIRE achieves higher final HP reduction and win rate than both in all four conditions. All methods start from the same SFT checkpoint. We evaluate PPO at the ten-round main budget and at thirty rounds because critic fitting was difficult in the multi-boss setting. The latter is an extended-budget comparison. Each checkpoint is evaluated in 20 episodes per condition.

MethodRoundsWind (50/600)Bear (50/600)Wind (native)Bear (native)
PPO1053.43 / 1057.71 / 1540.37 / 044.20 / 0
PPO3056.12 / 1062.90 / 2044.12 / 050.13 / 5
HG-DAgger1062.39 / 2068.56 / 3053.36 / 052.58 / 5
FIRE1070.95 / 3575.07 / 4058.57 / 1059.76 / 20

PPO improves with more training, but FIRE at ten rounds achieves higher HP reduction and win rate in all four conditions. The results are in Appendix B.1, Table 10 (p. 31).

HG-DAgger uses approximately 250 minutes of human effort, matching FIRE. The expert continuously monitors the agent and takes over when correction is needed. Monitoring and takeover exhaust this budget by the end of Round 3 and produce about 35 minutes of intervention video. We then stop new interventions and continue training with the collected corrections, mixed by boss with SFT demonstrations. This baseline does not use VLM failure labels or recording requests.

The ten-round comparisons use approximately 770 minutes of regular rollout across seven bosses. FIRE also uses about 90 minutes of diagnostic interaction with the fixed SFT policy before training, which we report separately. Evaluations are performed by the model alone, without human takeover.

We also report the intermediate checkpoint at Round 3, when the HG-DAgger human budget is exhausted, in Appendix B.2, Table 12 (p. 32). HG-DAgger has higher HP reduction than FIRE in three conditions at that point. FIRE continues to improve and achieves better final results in all four conditions. This makes the benefit of targeted recovery collection visible over the course of training.

AuthorsOpenReview ↗

Response to Reviewer hDyE(Part 3)

W3. Reliability across independent training seeds

All experiments appear to be conducted with a single random seed, and the paper does not report standard deviations or confidence intervals.

Across three independent RL training seeds, FIRE achieves higher mean HP reduction and win rate than Static recovery in all four conditions. We added two runs per method from the same selected SFT checkpoint and evaluated each Round-10 checkpoint in 20 episodes per boss and attribute setting. The table reports mean ± sample SD. HP and WR denote HP reduction and win rate (%).

ConditionMethodHPWR
Black Wind King 50/600FIRE69.74 ± 3.4230.00 ± 5.00
Black Wind King 50/600Static60.88 ± 2.2821.67 ± 2.89
Black Bear Guai 50/600FIRE73.33 ± 2.5638.33 ± 2.89
Black Bear Guai 50/600Static68.20 ± 2.5230.00 ± 5.00
Black Wind King nativeFIRE58.14 ± 3.3811.67 ± 2.89
Black Wind King nativeStatic52.25 ± 2.061.67 ± 2.89
Black Bear Guai nativeFIRE61.68 ± 2.0021.67 ± 2.89
Black Bear Guai nativeStatic53.26 ± 3.038.33 ± 2.89

The following approximate 95% confidence intervals for mean HP reduction use the three run-level means and a Student-t distribution with two degrees of freedom: mean ± 4.303 × SD / √3.

ConditionMethod95% interval
Black Wind King 50/600FIRE[61.25, 78.22]
Black Wind King 50/600Static[55.23, 66.53]
Black Bear Guai 50/600FIRE[66.97, 79.69]
Black Bear Guai 50/600Static[61.94, 74.46]
Black Wind King nativeFIRE[49.75, 66.54]
Black Wind King nativeStatic[47.13, 57.37]
Black Bear Guai nativeFIRE[56.71, 66.66]
Black Bear Guai nativeStatic[45.73, 60.80]

The original Figure 3 contains one training run per method. The new results measure variation across three training runs at the final checkpoint, with evaluation noise from the 20 episodes also contributing to that variation. PPO at each budget, HG-DAgger, and the fixed-label control are reported with one training run. The comparison tables use the original FIRE and Static runs. These details and the uncertainty estimates are now in Appendix B.4, Tables 14–15 (p. 33).

To show evaluation variability within each checkpoint, Appendix B.6, Figure 19 (p. 35) plots episode outcomes in 10-percentage-point bins, with wins shown separately and the three runs distinguished in each bar. Across 60 episodes per method and condition, FIRE wins 18, 23, 7, and 13 games, compared with 13, 18, 1, and 5 for Static recovery, in the condition order used above.

Reviewer_T1kHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FIRE uses evolving policy failures to guide human recovery demonstrations for a compact, pixel-based combat agent. A VLM coach diagnoses failures, and post-training combines hybrid-reward RL, curated replay, and supervised recovery learning. Experiments show improvements across seven in-domain bosses, with matched-video-budget controls supporting targeted recovery collection and additional behavior and held-out evaluations.

Critical strengths
  1. The work demonstrates a practical closed-loop data-collection system in continuous, real-time combat. A 2B actor improves HP reduction over SFT in all 14 in-domain configuration settings, with coaching confined to training.

  2. The successful-demonstration and static-recovery controls retain the main optimization components and match demonstration duration and staged availability. These controls and recovery-behavior measurements, provide useful evidence beyond an aggregate comparison against SFT.

Critical weaknesses
  1. The reliability of the core recovery gains is unclear: Figure 3 reports 20 evaluation episodes per point without confidence intervals or variation across independent training runs. Final win-rate gains over static recovery are only 5–10 percentage points (1–2 wins), so their robustness matters to the central claim.

  2. The static comparison changes both the failure-label set and recording allocation: static recovery samples initial modes uniformly, whereas FIRE prioritizes uncovered/frequent modes. It therefore does not isolate adaptation to newly emerging failures. Equal video duration also does not establish equal human effort (250 versus 170 minutes for FIRE and successful demonstrations).

  3. Figures 1 and 2 look AI-generated. I suggest redrawing them with a drawing tool for a more polished and professional presentation.

Questions for authors
  1. How many independent training runs underlie Figure 3? Please provide uncertainty for the core recovery comparisons, distinguishing evaluation variability from training variability.

  2. Does FIRE still improve over a control using its recording-priority rule restricted to initial failure labels? What is static recovery's total human collection effort?

5 comments
AuthorsOpenReview ↗

Response to Reviewer T1kH(Part 1)

Thank you for your careful review. We have added the new experiments and analyses to Appendix B of the revised PDF, with the additions marked in green.

W1. Reliability of the recovery gains

The reliability of the core recovery gains is unclear: Figure 3 reports 20 evaluation episodes per point without confidence intervals or variation across independent training runs.

Figure 3 uses one training run per method, with 20 evaluation episodes at each point. We have now added independent training runs and report their variation separately from the episode-level distributions.

Across three independent RL training seeds, FIRE achieves higher mean HP reduction and win rate than Static recovery in all four conditions. We added two runs per method from the same selected SFT checkpoint and evaluated each Round-10 checkpoint in 20 episodes per boss and attribute setting. The table reports mean ± sample SD. HP and WR denote HP reduction and win rate (%).

ConditionMethodHPWR
Black Wind King 50/600FIRE69.74 ± 3.4230.00 ± 5.00
Black Wind King 50/600Static60.88 ± 2.2821.67 ± 2.89
Black Bear Guai 50/600FIRE73.33 ± 2.5638.33 ± 2.89
Black Bear Guai 50/600Static68.20 ± 2.5230.00 ± 5.00
Black Wind King nativeFIRE58.14 ± 3.3811.67 ± 2.89
Black Wind King nativeStatic52.25 ± 2.061.67 ± 2.89
Black Bear Guai nativeFIRE61.68 ± 2.0021.67 ± 2.89
Black Bear Guai nativeStatic53.26 ± 3.038.33 ± 2.89

The following approximate 95% confidence intervals for mean HP reduction use the three run-level means and a Student-t distribution with two degrees of freedom: mean ± 4.303 × SD / √3.

ConditionMethod95% interval
Black Wind King 50/600FIRE[61.25, 78.22]
Black Wind King 50/600Static[55.23, 66.53]
Black Bear Guai 50/600FIRE[66.97, 79.69]
Black Bear Guai 50/600Static[61.94, 74.46]
Black Wind King nativeFIRE[49.75, 66.54]
Black Wind King nativeStatic[47.13, 57.37]
Black Bear Guai nativeFIRE[56.71, 66.66]
Black Bear Guai nativeStatic[45.73, 60.80]

The new results measure variation across three training runs at the final checkpoint, with evaluation noise from the 20 episodes also contributing to that variation. PPO at each budget, HG-DAgger, and the fixed-label control are reported with one training run. The comparison tables use the original FIRE and Static runs. These details and the uncertainty estimates are now in Appendix B.4, Tables 14–15 (p. 33).

To show evaluation variability within each checkpoint, Appendix B.6, Figure 19 (p. 35) plots episode outcomes in 10-percentage-point bins, with wins shown separately and the three runs distinguished in each bar. Across 60 episodes per method and condition, FIRE wins 18, 23, 7, and 13 games, compared with 13, 18, 1, and 5 for Static recovery, in the condition order used above.

AuthorsOpenReview ↗

Response to Reviewer T1kH(Part 2)

W2. Separating label adaptation from recording priority

Does FIRE still improve over a control using its recording-priority rule restricted to initial failure labels?

FIRE outperforms the fixed-label priority control in both HP reduction and win rate in all four conditions. This control retains VLM diagnosis and FIRE’s recording-priority rule while keeping the initial failure vocabulary fixed. Label assignments, statistics, and priorities continue to update, but no new labels are added. We evaluate Round 10 in 20 episodes per condition.

In the tables below, Wind and Bear denote Black Wind King and Black Bear Guai. Each performance cell reports HP reduction / win rate (%).

MethodWind (50/600)Bear (50/600)Wind (native)Bear (native)
Static recovery63.27 / 2568.88 / 3054.46 / 553.50 / 10
Fixed labels + priority64.19 / 2570.43 / 3054.31 / 056.86 / 10
FIRE70.95 / 3575.07 / 4058.57 / 1059.76 / 20

FIRE improves HP reduction over this control by 6.76, 4.64, 4.26, and 2.90 percentage points, with a ten-point win-rate gain in every condition. Priority alone improves HP reduction over Static recovery in three conditions, but does not improve win rate. The additional gains from FIRE support expanding the failure vocabulary as the policy develops, beyond reallocating collection within the initial categories. We have added this control and its results to Appendix B.3, Table 13 (p. 32).

Total human effort for Static recovery

What is static recovery's total human collection effort?

Static recovery required approximately 250 minutes of human effort, comparable to FIRE. Both collect about 84 minutes of demonstrations. The remaining time includes setting up the situations needed to demonstrate recovery from the targeted errors. Successful-demonstration collection takes less preparation and required about 170 minutes for the same video duration.

MethodHuman effort (min)Demonstration video (min)
FIRE25084
Static recovery25084
Successful demonstrations17084
HG-DAgger25035

We now report human effort alongside video duration in Appendix B.1, Table 11 (p. 31). The HG-DAgger cost includes the time spent continuously monitoring the agent, which explains why the same human budget produces a shorter demonstration video.

W3. Presentation of Figures 1 and 2

Figures 1 and 2 look AI-generated. I suggest redrawing them with a drawing tool for a more polished and professional presentation.

Thank you for the suggestion. Figures 1 and 2 were manually drawn and assembled in PowerPoint, rather than generated by an image model. AI tools were used only for layout and color suggestions. We agree that their visual presentation can be improved and will simplify the figures by reducing decorative elements and emphasizing the key workflow and technical components.

AuthorsOpenReview ↗

Response to Reviewer T1kH(Part 3)

Q1. Independent training runs and uncertainty

How many independent training runs underlie Figure 3? Please provide uncertainty for the core recovery comparisons, distinguishing evaluation variability from training variability.

Figure 3 uses one training run per method. We have added two independent RL runs for both FIRE and Static recovery, giving three runs per method at Round 10. As detailed in W1 above, Tables 14–15 report variation across training runs and confidence intervals, while Figure 19 shows the episode-level distributions within those runs.

Q2. Fixed-label priority control and Static recovery effort

Does FIRE still improve over a control using its recording-priority rule restricted to initial failure labels? What is static recovery's total human collection effort?

Yes. FIRE improves HP reduction and win rate over the fixed-label priority control in all four conditions, with a ten-percentage-point win-rate gain in each. Static recovery required approximately 250 minutes of human effort to collect 84 minutes of demonstrations. The control and cost breakdown are given in W2 above and in Appendix B.1 and B.3, Tables 11 and 13.

Reviewer_SGpmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an algorithm to control real time agents in combat video games. They first train a base model using supervised learning on human demonstrations, and then perform rounds of guided human data collection where they first roll out their policy and then use a VLM to identify and classify failures and then prompt humans with specific kinds of recovery behavior they wish to have their learned policy exhibit and collect these demos in the video game environment. The paper evaluates mostly on black mith wukong across different bosses and characters.

Critical strengths

I think that the motivation for their methodology makes a lot of sense, as well as the motivation to be able to target and guide data collection towards specific recovery behaviors. They empirically demonstrate it in multiple different video game environments and show strong gains over standard supervised learning with an increased expert demonstration set.

Critical weaknesses

I think comparing directly to dagger or hg-dagger given the fixed human rollout budget would help show the improvements of your method over other similarly motivated approaches towards guiding human data collection. These methods are discussed in the related works section, but they address the same train test time mismatch and this comparison would help clarify the benefits of including the VLM based selection and guidance module.

I also think that adding in statistical analysis with confidence intervals or increasing the number of eval episodes from 20 would help strengthen the paper as it is hard to tell if the smaller improvements over the balinese are significant without this.

Questions for authors

How does the recovery demonstrations change based on the familiarity of the human data collectors towards the video game and the given system?

What is preventing the success rates from saturating these tasks? Is this due to the fac that it is hard for an expert human to achieve perfect success?

5 comments
AuthorsOpenReview ↗

Response to Reviewer SGpm(Part 1)

Thank you for your constructive review. We have added new experiments and analyses to Appendix B of the revised manuscript, marked in green.

W1. Comparison with HG-DAgger under matched human effort

I think comparing directly to dagger or hg-dagger given the fixed human rollout budget would help show the improvements of your method over other similarly motivated approaches towards guiding human data collection.

We added an HG-DAgger baseline under matched human effort. FIRE achieves higher final HP reduction and win rate in all four evaluated conditions.

Both methods start from the same SFT checkpoint and use approximately 250 minutes of human effort. In HG-DAgger, an expert continuously monitors the agent and intervenes when correction is needed. This budget is exhausted by Round 3, yielding approximately 35 minutes of intervention video. Training then continues to Round 10 using these corrections and original SFT demonstrations, without VLM failure diagnosis or targeted recording requests.

All methods complete ten rounds of regular RL interaction (approximately 770 minutes across seven bosses). FIRE additionally uses approximately 90 minutes of fixed-SFT diagnostic rollouts, reported separately. Evaluations are performed without human intervention.

Wind and Bear denote Black Wind King and Black Bear Guai. Each cell reports HP reduction / win rate (%).

MethodRoundsWind (50/600)Bear (50/600)Wind (native)Bear (native)
SFT057.76 / 1557.31 / 2039.10 / 031.83 / 0
Static recovery358.44 / 1559.96 / 2542.87 / 038.44 / 0
FIRE360.79 / 1563.86 / 3043.28 / 041.49 / 0
HG-DAgger361.44 / 1564.25 / 2543.02 / 045.12 / 0
Static recovery1063.27 / 2568.88 / 3054.46 / 553.50 / 10
FIRE1070.95 / 3575.07 / 4058.57 / 1059.76 / 20
HG-DAgger1062.39 / 2068.56 / 3053.36 / 052.58 / 5

HG-DAgger initially achieves higher HP reduction in three conditions at Round 3. However, FIRE surpasses HG-DAgger in all four conditions by Round 10, improving HP reduction by 5.21–8.56 percentage points and win rate by 10–15 points. These results support the advantage of directing human effort toward evolving policy failures rather than relying on direct intervention. See Appendix B.1–B.2, Tables 10–12 (pp. 31–32).

AuthorsOpenReview ↗

Response to Reviewer SGpm(Part 2)

W2. Uncertainty in the recovery gains

I also think that adding in statistical analysis with confidence intervals or increasing the number of eval episodes from 20 would help strengthen the paper as it is hard to tell if the smaller improvements over the balinese are significant without this.

Across three independent RL training seeds, FIRE achieves higher mean HP reduction and win rate than Static recovery in all four conditions. We added two runs per method from the same selected SFT checkpoint and evaluated each Round-10 checkpoint in 20 episodes per boss and attribute setting. The table reports mean ± sample SD. HP and WR denote HP reduction and win rate (%).

ConditionMethodHPWR
Black Wind King 50/600FIRE69.74 ± 3.4230.00 ± 5.00
Black Wind King 50/600Static60.88 ± 2.2821.67 ± 2.89
Black Bear Guai 50/600FIRE73.33 ± 2.5638.33 ± 2.89
Black Bear Guai 50/600Static68.20 ± 2.5230.00 ± 5.00
Black Wind King nativeFIRE58.14 ± 3.3811.67 ± 2.89
Black Wind King nativeStatic52.25 ± 2.061.67 ± 2.89
Black Bear Guai nativeFIRE61.68 ± 2.0021.67 ± 2.89
Black Bear Guai nativeStatic53.26 ± 3.038.33 ± 2.89

The following approximate 95% confidence intervals for mean HP reduction use the three run-level means and a Student-t distribution with two degrees of freedom: mean ± 4.303 × SD / √3.

ConditionMethod95% interval
Black Wind King 50/600FIRE[61.25, 78.22]
Black Wind King 50/600Static[55.23, 66.53]
Black Bear Guai 50/600FIRE[66.97, 79.69]
Black Bear Guai 50/600Static[61.94, 74.46]
Black Wind King nativeFIRE[49.75, 66.54]
Black Wind King nativeStatic[47.13, 57.37]
Black Bear Guai nativeFIRE[56.71, 66.66]
Black Bear Guai nativeStatic[45.73, 60.80]

The original Figure 3 contains one training run per method. The new results measure variation across three training runs at the final checkpoint, with evaluation noise from the 20 episodes also contributing to that variation. PPO at each budget, HG-DAgger, and the fixed-label control are reported with one training run. The comparison tables use the original FIRE and Static runs. These details and the uncertainty estimates are now in Appendix B.4, Tables 14–15 (p. 33).

To show evaluation variability within each checkpoint, Appendix B.6, Figure 19 (p. 35) plots episode outcomes in 10-percentage-point bins, with wins shown separately and the three runs distinguished in each bar. Across 60 episodes per method and condition, FIRE wins 18, 23, 7, and 13 games, compared with 13, 18, 1, and 5 for Static recovery, in the condition order used above.

AuthorsOpenReview ↗

Response to Reviewer SGpm(Part 3)

Q1. How collector familiarity affects the demonstrations

How does the recovery demonstrations change based on the familiarity of the human data collectors towards the video game and the given system?

Our collectors are experienced players who can consistently defeat the evaluated bosses under native attributes, including five consecutive wins. Their expertise enables them to recreate difficult combat situations and demonstrate appropriate recovery behaviors.

FIRE does not require collectors to diagnose policy failures themselves. The VLM identifies failures and generates specific recovery requests, which experts execute and record. Familiarity with FIRE therefore mainly involves understanding these requests and translating them into demonstrations.

Original SFT and recovery demonstrations were collected by experts with comparable proficiency. As collectors were already familiar with these bosses, substantial improvement in game proficiency during collection was unlikely. We have not separately evaluated the effect of varying collector expertise.

Q2. Why model success does not reach the expert level

What is preventing the success rates from saturating these tasks? Is this due to the fac that it is hard for an expert human to achieve perfect success?

The remaining performance gap is not primarily due to the difficulty experts face in defeating these bosses. Our experienced collectors can consistently defeat them, including achieving five consecutive wins. The human reference in Table 1 consists of novice players and therefore does not represent expert-level performance.

A likely limitation is real-time control latency. Humans observe gameplay continuously, whereas our actor receives visual observations at 5 Hz while the game runs at 60 FPS. Action generation and transmission introduce additional delays, making timing-sensitive dodging and counterattacking more challenging. The remaining gap likely reflects these control constraints alongside imperfect learned policies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.