- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes a JAX acceleration environment tailored for Chinese Standard Mahjong.
Critical strengths
The engineering contribution of the paper is relatively significant
Critical weaknesses
-
The writing of the paper is extremely poor, containing numerous expressions that are difficult for humans to comprehend. I find it challenging to grasp the main points.
-
The paper lacks novelty and focuses on a very narrow research topic. Mahjong is already a minor topic, and Chinese Standard Mahjong is not important. I can hardly find any relevant top-tier conference work based on Chinese Standard Mahjong.
-
I don't believe that it's extremely difficult to create a correct MCR environment in Sep 2026. The contribution of the paper fails to meet the standards of a top conference.
Questions for authors
-
Did this work propose any general and scalable algorithms or engineering techniques?
-
Did you conduct experiments on multiple hardware and multiple seeds?
-
You have chosen "datasets and benchmarks" as your topic. What contributions have you made to datasets or benchmarks?
3 comments
Reviewer ySgw, please respond to authors today
Reviewer ySgw,
Today is the last day of the discussion period. The authors have prepared a detailed response to your review. Please reply to them as soon as possible to ensure there is a chance for back and forth discussion before the discussion period ends.
AC
Dear Area Chair 1NKw,
The author admits all weaknesses. This paper has nothing to discuss further.
Reviewer ySgw
Answers to your three questions, and what we concede
Thank you for the review. We agree on the writing: the paper is hard to follow, and that is ours to fix.
On importance we will not argue. Little work at top venues uses this ruleset, as you say. The one concrete finding we would point to is in Section 1 and Figure 2. A piece of code that, as Section 1 says, is widely copied in open-source agents for this game (it turns the game state into the agent's inputs and legal moves) disagrees with the official judge on 118 of the 12,088 winning hands in the corpus and would refuse 20 legal wins. Nothing crashes when it does. So we do not claim that a correct environment is hard to write. We claim that an incorrect one is easy to write without noticing, and that correctness should be checked against the judge's own records. Our own work is not exempt. After the reviews we checked our code and found five ways in which our training and evaluation setup departs from the game as the judge plays it; they are listed in our reply to Reviewer RxyH.
Q1. No new algorithm. The engine follows the design of Pgx, as Section 2 says. The part that could carry over to other games scored by an external rulebook is the replay check of Section 4: replay judged logs, keep one pass-or-fail row per game, and report which rule situations the logs cover. The paper states it only for other implementations of this game, and we have not applied it to a second one. It also needs tests for what logs cannot reach. Ours cover two such cases and, as the reply to Reviewer RxyH shows, miss others.
Q2. No. Timings come from one laptop GPU (the environment and the profile) and a desktop GPU (the training runs), with one machine per row and one timing per batch size (Table 5, Figure 3). Fine-tuning has one training seed, the from-scratch runs have two per condition, and the study of the value loss has one. The paper labels the fine-tuning result exploratory for that reason (Section 6.1).
Q3. Three things: the environment; the replay check against 12,288 judged games with a record per game, plus a 221-game subset that contains every rare event class of Table 2; and three graded tasks with an evaluation protocol (Section 6.1). Two limits apply. The corpus is an existing public test set that we do not redistribute. The imitation policy's weights are not released, so one of the three tasks cannot be rerun by others (Section 7).