acceptodds
Under review as a conference paper at ICLR 2027

A GPU Environment for Chinese Standard Mahjong with a Judge-Replay Fidelity Contract and Reference RL Workloads

Abstract

Accelerator-native simulators have made reinforcement learning (RL) on board games cheap, but when a game's payoff is fixed by an external rulebook, a fast simulator is useful only if it is demonstrably faithful: a mis-scored hand still yields a plausible learning curve. We present a GPU-native JAX environment for Chinese Standard Mahjong (MCR), a four-player imperfect-information game in which a win is legal only if it scores at least eight fan (pattern points) in an 81-pattern catalogue, with a fidelity contract that replays 12,288 adjudicated competition games and requires exact agreement on every judge event, per-seat win-eligibility value and final score. The reference engine that the JAX environment mirrors passes all 12,288 games on every field and the JAX environment reproduces their events and final scores (not the win-eligibility values), each with a per-game record; we also report the corpus's rule coverage. We find that the simulator is not what limits RL training: it steps up to 6.9M transitions per second on one laptop GPU, while a training iteration spends 64% of its time in the policy network. Reference workloads show the benchmark is hard to learn. From scratch, reward shaping teaches agents to reach a ready hand, but legal win rates never exceed 2.8% and end at zero. Fine-tuning a strong imitation policy leaves its win rate unchanged and raises score per game by 0.33 points (95% CI [0.05, 0.60]), an exploratory result from one training run.

Primary area:
datasets and benchmarks
Keywords:
reinforcement learning environment, JAX, Mahjong, imperfect-information games, benchmark validation, accelerator-native simulation, RL fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ySgwOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a JAX acceleration environment tailored for Chinese Standard Mahjong.

Critical strengths

The engineering contribution of the paper is relatively significant

Critical weaknesses
  • The writing of the paper is extremely poor, containing numerous expressions that are difficult for humans to comprehend. I find it challenging to grasp the main points.

  • The paper lacks novelty and focuses on a very narrow research topic. Mahjong is already a minor topic, and Chinese Standard Mahjong is not important. I can hardly find any relevant top-tier conference work based on Chinese Standard Mahjong.

  • I don't believe that it's extremely difficult to create a correct MCR environment in Sep 2026. The contribution of the paper fails to meet the standards of a top conference.

Questions for authors
  1. Did this work propose any general and scalable algorithms or engineering techniques?

  2. Did you conduct experiments on multiple hardware and multiple seeds?

  3. You have chosen "datasets and benchmarks" as your topic. What contributions have you made to datasets or benchmarks?

3 comments
AuthorsOpenReview ↗

Answers to your three questions, and what we concede

Thank you for the review. We agree on the writing: the paper is hard to follow, and that is ours to fix.

On importance we will not argue. Little work at top venues uses this ruleset, as you say. The one concrete finding we would point to is in Section 1 and Figure 2. A piece of code that, as Section 1 says, is widely copied in open-source agents for this game (it turns the game state into the agent's inputs and legal moves) disagrees with the official judge on 118 of the 12,088 winning hands in the corpus and would refuse 20 legal wins. Nothing crashes when it does. So we do not claim that a correct environment is hard to write. We claim that an incorrect one is easy to write without noticing, and that correctness should be checked against the judge's own records. Our own work is not exempt. After the reviews we checked our code and found five ways in which our training and evaluation setup departs from the game as the judge plays it; they are listed in our reply to Reviewer RxyH.

Q1. No new algorithm. The engine follows the design of Pgx, as Section 2 says. The part that could carry over to other games scored by an external rulebook is the replay check of Section 4: replay judged logs, keep one pass-or-fail row per game, and report which rule situations the logs cover. The paper states it only for other implementations of this game, and we have not applied it to a second one. It also needs tests for what logs cannot reach. Ours cover two such cases and, as the reply to Reviewer RxyH shows, miss others.

Q2. No. Timings come from one laptop GPU (the environment and the profile) and a desktop GPU (the training runs), with one machine per row and one timing per batch size (Table 5, Figure 3). Fine-tuning has one training seed, the from-scratch runs have two per condition, and the study of the value loss has one. The paper labels the fine-tuning result exploratory for that reason (Section 6.1).

Q3. Three things: the environment; the replay check against 12,288 judged games with a record per game, plus a 221-game subset that contains every rare event class of Table 2; and three graded tasks with an evaluation protocol (Section 6.1). Two limits apply. The corpus is an existing public test set that we do not redistribute. The imitation policy's weights are not released, so one of the three tasks cannot be rerun by others (Section 7).

Area_Chair_1NKwOpenReview ↗

Reviewer ySgw, please respond to authors today

Reviewer ySgw,

Today is the last day of the discussion period. The authors have prepared a detailed response to your review. Please reply to them as soon as possible to ensure there is a chance for back and forth discussion before the discussion period ends.

AC

Dear Area Chair 1NKw,

The author admits all weaknesses. This paper has nothing to discuss further.

Reviewer ySgw

Reviewer_RxyHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a JAX environment for Chinese Standard Mahjong (MCR). The game state is a fixed-size tensor tree; transitions are dispatched with lax.switch; and scoring runs on the host via a callback to the open-source fan calculator. The other contributions are:

  • Fidelity contract. Engines replay 12,288 adjudicated games from the IJCAI-2026 competition final (512 walls × 24 seat orders). A pure-Python reference engine reproduces every judge field byte-exactly. The JAX engine reproduces events and final scores, but not the per-seat win-eligibility values, wall counts, or request strings.
  • A bug finding. A widely copied feature layer mis-scores 118 of 12,088 winning hands.
  • Throughput by level. The environment alone reaches 6.9M steps/s on a laptop GPU. Training is bound by the policy network (64% of iteration time).
  • Three benchmark tasks with baseline runs. From scratch, agents reach ready hands but almost never win legally. KL-regularized PPO fine-tuning of an imitation anchor leaves win rate unchanged and raises score by 0.33 points per game (95% CI [0.05, 0.60]) in a single run.
Critical strengths
  1. Fidelity is checked against the adjudicator, not a second implementation. Game-by-game replay with per-game records, a "golden" subset covering rare events (robbed kongs, competing wins), and the documented feature-layer bug together make a strong case that simulator correctness must be measured against the judge. This is a useful methodological contribution for rule-defined games.
  2. Honest throughput reporting. Environment-only, rollout, and full-iteration rates are separated, the training bottleneck is identified, and the limits of the CPU comparison are stated.
  3. Careful evaluation methodology for the RL workload. The paired 15,000-game gate, the demonstration that small checkpoint evaluations can get the sign wrong, and the clear exploratory/confirmatory distinction are good practice.
  4. Good reproducibility materials. The comparator, per-game records, tests, logs, and a script that recomputes every number are provided.
Critical weaknesses
  1. The JAX engine — the artifact actually used for RL — is not validated on the fields that matter most for learning. Its replay checks events and final scores. It does not check the per-seat win-eligibility values or the legal-action information, whose agreement is "inferred from its construction, not measured" (Section 4). For an RL environment, the legality mask and the win-eligibility signal define the action space agents explore. Moreover, the on-device mask admits any structurally complete hand, and sub-8-fan declarations are handled as a forfeit (the declarer pays 24; others receive 8). This departs from the adjudicated rules exactly where learning agents explore and where replay of competent play gives no evidence. Please (a) extend the JAX comparator to the win-eligibility field, at least on the golden subset, and (b) state whether the forfeit payoff matches the official or Botzone penalty for a false win declaration, or justify the deviation.

  2. Rule coverage is incompletely characterized. The corpus comes from four finalist programs on 512 distinct walls. The paper does not report how many of the 81 fan patterns appear, although the records list adjudicated fans (Section 4). Rare terminal situations have single-digit counts. A per-pattern coverage table is straightforward to produce and important for a fidelity claim. Adding fuzzed or randomized play, checked against the judge (which can be queried), would test transitions that competent play never reaches.

  3. The case for an accelerator-native environment is weak for the reported workloads. With the 40-block policy, training runs at about 1.5k–6.4k steps/s and is bound by the policy forward pass. The environment step is 0.05% of iteration time, compared with millions of environment-only steps per second (Table 5). The only CPU baseline is a single-threaded validating replay checker, and linear scaling to 32 threads would already match the GPU environment (Section 5). The practical advantage claimed — no host–device transfer per step — is plausible but not measured. An end-to-end comparison of training throughput (a multi-process CPU environment such as EnvPool feeding the same GPU policy, against the in-scan JAX environment) is needed to support the first contribution.

  4. The reference workloads are thin, and the "hard to learn" conclusion is premature.

    • Fine-tuning uses one training seed, and the paper's own protocol requires three for a confirmatory claim.
    • From-scratch training uses a small network, terminal-only reward with γ = 1, and two seeds per condition.
    • None of the components known to matter in Mahjong AI are tried: oracle guiding, global reward prediction, self-play populations, search, or the larger-scale training of Suphx or ACH.
    • The finding that agents fail to win from scratch may therefore reflect the recipe rather than the benchmark. I suggest framing the workloads as sanity checks, or adding at least one stronger, established training approach as a baseline.
  5. Task I cannot be reproduced. The imitation anchor's weights are not released (Reproducibility Statement and Limitations), so others cannot run the improvement-beyond-imitation task against the same reference. For a benchmark contribution, releasing the anchor, or defining Task I against a publicly available policy, is essential.

  6. The scope is narrow relative to the venue. The environment covers one ruleset with single-hand games. Its design pattern follows Pgx and Mahjax (fixed-size state, masked phase dispatch), and the scorer is an existing open-source calculator called on the host. The main new elements are the MCR-specific transition logic and the validation contract. That is valuable, but the ML contribution is modest, and the paper would be better suited to a datasets-and-benchmarks track unless the workloads produce new insights.

  7. Minor issues.

    • Evaluation uses only seat 0 against three frozen anchors; seat-position effects are not examined.
    • The "widely copied feature layer" should be identified, by citation or footnote, so the community can fix it.
    • The paired gate records only aggregate component rates, so the score improvement cannot be decomposed (Section 6.4).
Questions for authors
  1. Can the JAX comparator be extended to per-seat win-eligibility values and legal-action sets on at least the golden subset? Do they agree with the judge?
  2. Does the illegal-win forfeit payoff (−24/+8) match the official MCR or Botzone false-win penalty? If not, how might it change learned behavior?
  3. How many of the 81 fan patterns does the corpus exercise, and which ones are never observed?
  4. What end-to-end training throughput does a multi-process CPU environment with the same GPU policy achieve, compared with the in-scan JAX environment?
  5. Will the anchor policy be released? Can Task I be run with at least three training seeds?
  6. Have you tried a stronger training recipe (e.g., self-play or oracle guiding) to test whether "hard to learn" is a property of the benchmark?
2 comments
AuthorsOpenReview ↗

Weakness 1 is right, and wider than the paper says; the forfeit does not match the judge; what is not available

Thank you for the careful review. We agree with all seven weaknesses. Checking Weakness 1 against our code, we found more than the paper admits, so that comes first. There is no revised PDF: what matters needs runs we have not done.

W1 / Q1. Not run, though it would measure a real gap. The JAX environment's own legality mask never marks the win action legal; in the workloads the training code adds it from a shape test (four melds and a pair, or seven pairs). So on win eligibility the JAX side does not mirror the reference engine, and Section 4 ("inferred from its construction") and the Limitations are wrong for that field. The replay does not test the mask either: it feeds the logged actions in and compares events and final scores.

We found five ways in which the workloads' game differs from the judged game. Only the forfeit is in the paper.

  1. Only those two shapes can win. The special hands (knitted hands, thirteen orphans) cannot be declared, so "any structurally complete hand" in Section 3 is wrong. In the 6,000-game subset whose winning hands we decomposed, 339 of 5,921 (5.7%) fit neither shape.
  2. The forfeit below 8 fan (−24, +8) is not the judge's penalty (Q2).
  3. After a granted Chi or Peng, the claimer's discard is set by a fixed rule for all four seats, not by the policy: the tile kind it holds most copies of before the claim, the claimed kind aside.
  4. That rule reads the hand before the claim, so after a Chi it can name a tile whose last copy the Chi has just used. The step then discards a tile the seat no longer holds: its count becomes −1 and a tile that does not exist enters the discards. Nothing catches it: the silent corruption Section 3 warns about, outside our two gates. We have not measured how often it occurs.
  5. The observation shows every seat the tile of another seat's concealed kong, which the judge does not send to the other seats (Section 3 says "exposed melds").

Beyond these five, each training run uses a single prevalent wind, while the corpus has all four. So Section 4's contract certifies the transition function and the scorer on logged games, not that the workloads play the judged game. They do not.

Q2. It does not match Botzone's. In that judge's public source a false win is treated like any illegal action: −30 for the offender, +10 for each other seat, and the game ends. Ours has the same form at 0.8 of the size; the paper gives no reason, and we did not check the printed rulebook. On behaviour: we never counted forfeits. A policy trained under the rule can raise its score at an unchanged win rate by declaring fewer hands below 8 fan, so the +0.33 points per game should not be read as better play. The released logs show forfeits happening: in one from-scratch seed, seat 0 never wins or deals in across 167 evaluations, yet its mean score is positive in 166 (median +0.37; the other seed +0.004), which can only be +8 from other seats' forfeits, at least 93 per 2,000 games at the median. For the anchor and the fine-tuned policy we have no count.

W2 / Q3. Not counted. The records list the fans, but our tabulation kept only totals and the tags of Table 2. We have not run generated play against the judge; it would catch items 1 and 4, which replay cannot.

W3 / Q4. Not measured. Our only CPU training figure is an earlier CPU-environment trainer at about 313 steps per second (a comment in the training code), not a matched measurement. One correction: Section 5 gives 8.6× at "the batch used in training (512)", but the main run used batch 256, where the ratio is 4.7×; 512 is the profiling and from-scratch batch.

W4 / Q6. Agreed. We tried no stronger recipe; the short run against past checkpoints (Section 6.4) says nothing. "Hard to learn" claims more than we showed: every workload number may reflect this setup and not the game. The from-scratch code also evaluates against frozen copies of the untrained initial network, which Section 6.4 does not say.

W5 / Q5. The weights are not in the anonymous supplement and we are not adding them during review, so Task I cannot be rerun by others today (Section 7); a later release is undecided. We ran one seed, not three, and label the result exploratory. We agree that a benchmark task needs a public reference policy.

W6. We submitted under datasets and benchmarks, and we agree that the ML contribution is modest.

W7. Seat 0 only: agreed, and Table 3 points to a seat effect: the anchor wins on a discard in 11.5% of games from seat 0 against a deal-in rate of 10.2%, while over four identical seats the two rates are equal on average. Feature layer: the class is FeatureAgent; its faulty line sets the last-tile flag by shownTiles[winTile] == 4, which misses a self-drawn tile (the supplement's test-set script shows the fix). We have not traced it to an original source we could cite. Decomposition: agreed; forfeits were not recorded either.

Area_Chair_1NKwOpenReview ↗

Reviewer RxyH, please respond to authors today

Reviewer RxyH,

Today is the last day of the discussion period. The authors have prepared a detailed response to your review. Please reply to them as soon as possible to ensure there is a chance for back and forth discussion before the discussion period ends.

AC

Reviewer_C9hZOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces an environment suitable for training AI models to play Chinese Standard Mahjong. The environment has nice engineering properties, including that it can be run on the GPU alongside the model, so data does not constantly need to be moved from CPU to GPU and back. The paper shows that the environment is hard, in that baseline RL methods cannot improve their win rate through training.

Critical strengths
  1. Chinese Standard Mahjong is an important game in the pantheon of human-played games. So, it makes sense that there should be a practical simulator for training and evaluating AI models in the game.
Critical weaknesses
  1. The contribution appears to be an incremental engineering contribution, with limited evidence of how it could be important for AI research.
  2. The text is difficult to read. It is difficult to see the point the text is trying to convey throughout. I am mostly convinced that the implemented environment works as a valid implementation of the Mahjong rules, but it is hard to parse any additional valuable ideas from the text.
Questions for authors
  1. Is it possible to motivate the environment from a research perspective? E.g., is there some critical behavior missing from RL systems that could be resolved if they could learn Chinese Standard Mahjong?
  2. Does the environment change the scale of what kinds of experiments are possible in this game? E.g., was it previously prohibitively costly to train Mahjong models for most researchers, but your new environment now makes it possible? Do you have evidence for this?
2 comments
AuthorsOpenReview ↗

What the paper does and does not show about research use and cost

Thank you for the review. We agree with both weaknesses: the contribution is an engineering one, and the text is hard to read.

Q1. We cannot name a behaviour missing from RL systems that this game would fix, and the paper does not show one. What the game offers is a win condition that a complete hand does not meet by itself: the hand must also score at least 8 fan (pattern points) from a catalogue of 81 patterns. In our from-scratch runs with reward shaping, agents reach a ready hand (one tile short of complete) in 57% and 64% of the games that end without a winner, yet their legal win rate never exceeds 2.8% and ends at zero (Section 6.4). That is two seeds, one small network and one recipe, and the released code evaluates them against untrained opponents. It is also weaker than the abstract makes it sound. Checking our code after the reviews, we found five ways in which our training and evaluation setup departs from the game as the official judge plays it, four of them not stated in the paper; they are listed in our reply to Reviewer RxyH. "Hard to learn" is therefore shown at most for that setup, not for the game.

Q2. We have no measurement that shows it. The environment alone runs at up to 6.9M steps per second on a laptop GPU. With the 40-block policy, a full training iteration on that laptop GPU runs at 1,482 steps per second, and 64% of its time is the policy network (Section 5). The main fine-tuning run, on a desktop GPU at 6,358 steps per second, took 30.6 hours, evaluation included, for 545.6M steps (Section 6.2, Tables 5 and 6). The paper reports no measured comparison with a CPU environment feeding the same GPU policy, so we cannot say that these experiments were out of reach before. What the measurements support is narrower: with this environment, the simulator is not what limits training.

Area_Chair_1NKwOpenReview ↗

Reviewer C9hZ, please respond to authors today

Reviewer C9hZ,

Today is the last day of the discussion period. The authors have prepared a detailed response to your review. Please reply to them as soon as possible to ensure there is a chance for back and forth discussion before the discussion period ends.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.