acceptodds
Under review as a conference paper at ICLR 2027

Range-Calibrated GFlowNets

Abstract

Generative Flow Networks (GFlowNets) learn policies for sampling compositional objects in proportion to a specified terminal reward. Applying an inverse temperature to reward values controls concentration, but the same temperature can assign different probabilities to a top quantile on different reward landscapes. To specify target concentration, we propose Range-Calibrated GFlowNets (RC-GFN), which use exponential weights on ranks estimated against a policy-independent reference set. Under a uniform reference measure without score ties, the concentration parameter determines the population target's top-quantile mass in closed form, independently of terminal-space size and reward values. The empirical weights are invariant to strictly increasing score transformations and have a user-controlled log-range bound. For arbitrary reference measures and tied scores, we bound finite-reference target error independently of terminal-space size and separate it from policy-fitting error. Experiments on an enumerable environment examine concentration across landscapes and reference sizes. On molecular and biological sequence design tasks, RC-GFN is competitive with the tested temperature-conditional GFlowNets in both mode discovery and top-candidate reward, and outperforms a representative rank-based GFlowNet.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
GFlowNets, reward temperature, quantile invariance

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zbofOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose RC-GFN that uses frozen reference rewards and enable to concentration parameter an interpretable meaning. It derives population calibration and finite-reference error bounds, with experiments on HyperGrid and four molecular/sequence benchmarks.

Critical strengths

The motivation is clear, the method is simple, and the separation of reference error from policy-fitting error is useful. The mode-discovery improvements, particularly on sEH and RNA, are encouraging.

Critical weaknesses

Practical calibration remains uncertain. Table 5 shows a TFBind10 top-set mass of 0.292 under the learned policy versus 0.673 under the frozen target. Can the authors identify the main cause and test whether an alternative training objective or replay strategy reduces this gap? Related approaches such as SubTB (Madan et al., 2023) and RapTB (Wang et al., 2026) should be discussed as complementary training methods. Beyond current experiments, a controlled, enumerable setting such as, a bounded Game of 24. It can help assess how reference-set size and coverage affect calibration under sparse rewards. In particular, what happens when the reference set misses all/most of correct solutions, and how do reward ties affect the resulting concentration?

The empirical comparison could better isolate the benefit of rank calibration. Please clarify whether tuning budgets were comparable. Since GR-GFN also uses rank-derived targets, can the authors clarify why its comparison is limited to HyperGrid? Another downstream comparison will help support the method. If underperform on some benchmarks, diagnosis and analysis on such cases will also be helpful.

Questions for authors

See weakness part. I'm willing to raise my score if the authors solve my concerns.

2 comments
AuthorsOpenReview ↗

Response to Reviewer zbof (Part 1/2)

We thank the reviewer for the careful reading and constructive feedback. Our response rests on three additions to the revised manuscript. First, a concentration experiment calibrates the concentration parameter λ\lambda of RC-GFN and the inverse temperature β\beta of the temperature-controlled baselines before training to the same promised mass on the top set TqT_q (Section 4.3 "CONCENTRATION EXPERIMENT"). Second, a failure case analysis of TFBind10 tests interventions on the training objective, the replay strategy and other factors of the original setting (Appendix J "FAILURE CASE ANALYSIS"). Third, a sparse reward experiment in a bounded Game of 24 examines how the reference size, the coverage of the reference and reward ties affect calibration (Appendix K "SPARSE REWARD EXPERIMENT"). We address your concerns point by point below.

Respond to Critical Weaknesses 1: Large fitting error on TFBind10 and complementary training objectives and replay strategies.

We agree that RC-GFN shows a large fitting error on TFBind10 in the mode discovery experiment, and the above concentration experiment confirms this observation. This difficulty is shared by all tested methods. In the concentration experiment, the temperature-controlled baselines also fall far short of the promised mass on TFBind10, while RC-GFN comes closest to it at every promised mass (Table 2). We therefore analyze TFBind10 as a failure case in Appendix J “FAILURE CASE ANALYSIS”, with a series of interventions that each change one factor of the original setting (including the SubTB objective and a change of replay strategy suggested by the reviewer). Some of these interventions slightly reduce the fitting error, but none of them brings the mass of the policy on TqT_q close to that of its target. We then compare TFBind10 with TFBind8 [1,2], which is built in the same way and differs only in the sequence length and the transcription factor. The two reward landscapes are similarly rugged, but their promised masses are fulfilled very differently. A visualization of the two landscapes shows that the modes of TFBind10 are far more scattered across the design space, which helps explain its persistent policy-fitting gap. RC-GFN uses ranks rather than numerical reward gaps to specify its target weights. In the concentration experiment, its learned policy is generally more uniform within the top set TqT_q than the temperature-controlled baselines. Moreover, on TFBind10, RC-GFN places more mass on the mode set than the temperature-controlled baselines at all three promised masses and turns it into more discovered modes, as shown in Section 4.3 "CONCENTRATION EXPERIMENT".

In addition, we now discuss SubTB and RapTB as complementary training methods in Section 2 "RELATED WORK". RapTB is designed for language-model policies in which every prefix can terminate and receives its own reward, which does not hold in our tasks, where only complete molecules and sequences are evaluated. The original RC-GFN setting already trains with the prioritized replay training (PRT) of Shen et al. [3], and the replay intervention replaces it with uniform replay.

[1] Barrera, Luis A., et al. "Survey of Variation in Human Transcription Factors Reveals Prevalent DNA Binding Changes." Science, vol. 351, no. 6280, Mar. 2016, pp. 1450–54.

[2] Trabucco, Brandon, et al. "Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization." Proceedings of the 39th International Conference on Machine Learning, PMLR, June 2022, pp. 21658–76.

[3] Shen, Max W., et al. "Towards Understanding and Improving GFlowNet Training." Proceedings of the 40th International Conference on Machine Learning, PMLR, July 2023, pp. 30956–75.

Respond to Critical Weaknesses 2: Effect of reference size, coverage and ties under sparse rewards in a bounded Game of 24.

To assess how reference size and coverage affect the calibration of RC-GFN under sparse rewards, we designed a bounded Game of 24 based on the Expr24 task of RapTB (Appendix K "SPARSE REWARD EXPERIMENT"). We found that a larger reference lowers the reference error and brings the target closer to the promised mass. Excluding all correct expressions from the reference leaves the frozen target mass on the correct set nearly unchanged. A reference that misses most of them lies between full coverage and this case. Excluding also the incorrect expressions closest to 24 lowers this mass markedly, because these expressions then share the highest empirical rank with the correct set. Reward ties make the λ\lambda given by the no-tie formula (Section 3.4, Proposition 4) over-concentrate the target, while a λ\lambda solved from the tie-group sizes restores the promised mass in the target, and the learned policy follows it, although a fitting gap remains.

AuthorsOpenReview ↗

Response to Reviewer zbof (Part 2/2)

Respond to Critical Weaknesses 3: Comparability of tuning budgets across methods.

To make the tuning of RC-GFN and the temperature-controlled baselines comparable, we use the concentration experiment introduced above. Since λ\lambda and β\beta are both solved from the same promised mass before training, the two methods receive the same tuning budget. We then compare the mass each method delivers on TqT_q, how uniformly its policy spreads this mass within TqT_q and how many modes it discovers during training. Except on TFBind10, both RC-GFN and the baselines largely fulfill the promised masses. However, the policy of RC-GFN is more uniform within TqT_q, and RC-GFN discovers modes faster or discovers more modes on most tasks and promised masses (Section 4.3 “CONCENTRATION EXPERIMENT”).

Respond to Critical Weaknesses 4: Downstream comparison with GR-GFN and diagnosis of underperformance.

GR-GFN was designed for multi-objective optimization, which differs substantially from our single-objective tasks, and its code is not publicly available, so faithfully reproducing its details on our downstream tasks was not straightforward, and we therefore reproduced GR-GFN in a minimal form and tested it only on HyperGrid. We have now added GR-GFN as a baseline to Section 4.2 “MODE DISCOVERY EXPERIMENT”, and its implementation details are given in Appendix H.1.2 “IMPLEMENTATION DETAILS”. RC-GFN discovers more modes than GR-GFN on sEH, TFBind10 and RNA, whereas GR-GFN performs best on QM9. To diagnose this case, we computed the exact sampling mass that each trained policy places on the top set TqT_q of each task, where qq is the quantile that defines the mode set.

TaskTarget mass of RC-GFNRC-GFNGR-GFN
QM90.1810.188 ± 0.0170.454 ± 0.009
sEH0.1390.267 ± 0.0260.697 ± 0.038
TFBind100.1310.034 ± 0.0010.023 ± 0.000
RNA0.8850.764 ± 0.0080.182 ± 0.006

On QM9, the mass that RC-GFN places on TqT_q is close to its target mass. GR-GFN fits a cross-entropy toward the best candidate of each batch, which has no finite optimum, so its concentration keeps increasing during training and is set by the training length rather than by a parameter. Within this budget, GR-GFN places much more mass on TqT_q than RC-GFN on QM9, where the mode set covers a large part of TqT_q, so the higher concentration yields more modes. The same mechanism overshoots on sEH, where GR-GFN places most of its mass on TqT_q and discovers substantially fewer modes than RC-GFN. On TFBind10 and RNA, GR-GFN places less mass on TqT_q than RC-GFN and discovers fewer modes. Note that the mode discovery experiment does not require RC-GFN to deliver its target mass, and we designed a separate experiment in Section 4.3 "CONCENTRATION EXPERIMENT" to evaluate whether the promised concentration is delivered.

Reviewer_UnsuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper provides a new method to control target distribution concentration independently of numerical reward scales, reward spacing, and terminal space size N=∣X∣N = |\mathcal{X}|. Basically in standard GFlowNet concentration methods, we tend to apply an inverse temperature β\beta to raw rewards (most commonly R(x)βR(x)^\beta or eβR(x)e^{\beta R(x)}), which makes the resulting top-quantile sampling mass highly sensitive to the specific reward landscape and scale.

This method, which is called Range-Calibrated Generative Flow Networks (RC-GFN), resolves this by evaluating candidate scores against a policy-independent reference set Y=(Y1,…,Ym)∼iidμY = (Y_1, \dots, Y_m) \stackrel{\text{iid}}{\sim} \mu drawn and frozen before training. For any object x∈Xx \in \mathcal{X}, RC-GFN defines an empirical rank coordinate Φ^(x)=1m∑j=1m1{R(Yj)<R(x)}∈[0,1]\hat{\Phi}(x) = \frac{1}{m} \sum_{j=1}^m \mathbf{1}\{R(Y_j) < R(x)\} \in [0, 1]. The bounded exponential rank reward R^(x)=eλ(Φ^(x)−1)\hat{R}(x) = e^{\lambda (\hat{\Phi}(x) - 1)} induces the target distribution p^(x)=R^(x)/Z^\hat{p}(x) = \hat{R}(x) / \hat{Z}.

On the empirical side, the methodology is validated on synthetic HyperGrid environments (D=2, H=64; D=3, H=32) across indicator, cosine, and spread landscapes, showing similar concentration across disparate topographies. On molecular design (QM9, sEH) and biological sequence design (TFBind10, RNA), the authors show that RC-GFN is competitive with strong baselines (OP-GFN, Layer-GFN, Logit-GFN, GFN-β\beta, MARS, PPO, A2C, SQL).

Critical strengths

What I found interesting is that the authors found a closed-form concentration semantics and provided a landscape invariance, which is very useful for generative modeling, especially in the case of sampling methods. More specifically, the paper solves a key limitation of temperature-controlled GFlowNets, namely that temperature β\beta yields variable concentration depending on score scale and spacing, by introducing a rank-based coordinate Φ^(x)\hat{\Phi}(x). I noticed in Proposition 4 they established that under a uniform reference measure without ties, the population top-qq quantile mass p∗(Tq)=(1−e−λq)/(1−e−λ)p^*(T_q) = (1 - e^{-\lambda q})/(1 - e^{-\lambda}) is closed-form, strictly monotonic, and completely independent of numerical score values and state-space size NN. This gives λ\lambda a scale-invariant physical interpretation across diverse task landscapes.

Critical weaknesses

What I am concerned about is the choice of the reference set size mm in practical settings where evaluating the true oracle R(x)R(x) is expensive. The authors give sufficient reference sizes of order λ2\lambda^2 in general (Theorem A.3) and of order λ\lambda under a uniform reference without ties (Proposition A.6), and they count the reference calls in the 64,000 oracle budget, which is good. However, the empirical rank coordinate can only resolve quantiles down to about 1/m1/m, and the reference sizes used are m=1,200m = 1{,}200 on QM9 and m=4,500m = 4{,}500 on sEH, i.e. a resolution of roughly 8×10−48 \times 10^{-4} and 2×10−42 \times 10^{-4}. As my understanding in GFN, when qq is extremely small (let's say q<10−4q < 10^{-4} in a large design space like sEH with about 3.4×1073.4 \times 10^7 candidates), drawing an i.i.d. uniform reference set YY would require a much larger mm to hit the top set with high probability, and this would take a large part of the oracle budget.

Questions for authors

In practical settings where evaluating the true oracle R(x)R(x) is expensive, how do the authors recommend choosing mm relative to the total oracle call budget when qq is very small (below 1/m1/m for the reference sizes used in the experiments)? How do the results on sEH change if a part of the budget is moved from policy training to a larger reference set?

1 comment
AuthorsOpenReview ↗

Response to Reviewer Unsu

We thank the reviewer for the careful reading and constructive feedback. We address your concerns point by point below.

Respond to Critical Weaknesses 1: The empirical rank resolves quantiles only down to about 1/m1/m.

The empirical rank coordinate takes at most m+1m+1 distinct values. Candidates whose scores exceed the maximum reference score therefore receive the same transformed reward, even if their original scores differ. Under a uniform reference measure without score ties, Proposition A.7 shows that the expected fraction of the terminal space in this top bin is at most 1/(m+1)1/(m+1). This explains the resolution scale identified by the reviewer. However, 1/m1/m is not a strict lower bound on the quantiles that can be targeted, since the realized size of the top bin depends on the reference draw.

The limitation is that candidates within the same empirical rank bin cannot be distinguished by the transformed reward, regardless of how large λ\lambda becomes. We have clarified this point in Section 5 "DISCUSSION" and Appendix A.4 “REFERENCE RESOLUTION AND TIES”.

Respond to Critical Weaknesses 2: Hitting a very small top set requires a large reference.

For a top set TqT_q occupying a fraction qq of the terminal space, Proposition A.7 gives the probability that an iid uniform reference YY of size mm misses the entire set,

Pr⁡(Y∩Tq=∅)=(1−q)m.\Pr(Y\cap T_q=\varnothing)=(1-q)^m.

To hit this set with probability at least 1−δ1-\delta, the required size is approximately

m≳log⁡(1/δ)qm\gtrsim\frac{\log(1/\delta)}{q}

for small qq. We agree that this can be expensive when the target quantile is extremely small.

However, hitting TqT_q is not necessary for its members to receive the largest empirical reward. If the reference misses TqT_q, these candidates still receive the maximum weight when encountered during training. The difficulty is that other candidates above the reference maximum receive the same weight. Writing BmB_m for this unresolved top bin, Proposition A.7 gives

p^(Tq)≤∣Tq∣∣Bm∣,\hat p(T_q)\leq\frac{|T_q|}{|B_m|},

with equality as λ→∞\lambda\to\infty. Thus a reference miss does not make the top set inaccessible, but it can limit the mass assigned to it by the frozen target. This limitation is distinct from the accuracy of the reference approximation and from the difficulty of discovering the top candidates during training. We tested this in a controlled experiment where the terminal space can be enumerated (Appendix K "SPARSE REWARD EXPERIMENT", Table 15). Each candidate is an arithmetic expression, its reward decreases with the distance between its value and 24, and TqT_q is the set of expressions that evaluate to exactly 24. Removing all of TqT_q from the reference leaves the frozen target mass on TqT_q nearly unchanged. Removing also the expressions whose values are closest to 24 lowers it markedly, because these expressions then fall into BmB_m together with TqT_q.

Respond to Questions For Authors:

We choose the reference size by balancing the accuracy of the frozen target against the oracle calls available for policy training. Under a uniform reference measure without score ties, Proposition A.6 gives the sufficient condition m≥λ/(4ε2)m\geq\lambda/(4\varepsilon^2) for top-set RMSE at most ε\varepsilon. This holds whether or not the reference hits TqT_q, but it is not a necessary reference size. The general high-probability bound in Theorem A.3 also covers tied scores and nonuniform references. For example, consider a top set with q=10−4q=10^{-4} under a budget of 64,000 oracle calls, the budget of our QM9, sEH and TFBind10 experiments. With promised mass P=0.6P=0.6, the no-tie calibration gives λ≈9,163\lambda\approx9{,}163. The sufficient reference size is then about 229,000 for ε=0.1\varepsilon=0.1, which exceeds the budget, or 57,300 for ε=0.2\varepsilon=0.2, which leaves little of the budget for training. Reducing PP to 0.3 gives λ≈3,567\lambda\approx3{,}567 and a sufficient size of about 22,300 for ε=0.2\varepsilon=0.2. We give the selection rule in Section 3.6 "ALGORITHM" (Algorithm 1, Step 1) and discuss its assumptions in Appendix E "CHOOSING THE REFERENCE SIZE".

The bounds above provide reference-size guidelines, but the best allocation also depends on the policy-training budget. To examine whether a larger reference helps under a fixed budget, we moved part of the policy-training budget on sEH into the reference, with m=60λm=60\lambda, 100λ100\lambda and 200λ200\lambda (Appendix E). Compared with m=30λm=30\lambda in the main experiments, a larger reference further lowers the reference error, but the number of discovered modes decreases as the reference takes a larger share of the budget. At m=30λm=30\lambda the reference error is already much smaller than the policy-fitting error, so further reference calls reduce the budget for policy training without improving the result. These results favor a moderate reference once its approximation error is small relative to policy-fitting error, with the remaining calls allocated to training.

Reviewer_zmRTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Range-Calibrated GFlowNets (RC-GFN), which replace value-based terminal rewards with exponential weights of empirical ranks computed against a frozen, policy-independent reference set. Under a uniform reference measure and without score ties, the population target assigns mass (1−e−λq)/(1−e−λ)(1-e^{-\lambda q})/(1-e^{-\lambda}) to the top-qq fraction of candidates. This gives the concentration parameter an interpretation independent of numerical reward values. The paper establishes finite-reference approximation bounds, analyzes reference resolution and ties, and separates reference error from policy-fitting error. Experiments examine concentration control on HyperGrid and candidate discovery on four molecular and biological sequence tasks.

Critical strengths

The paper proposes an exponential rank-based reward in place of conventional temperature scaling acting direct on given reward function. Adjusting λ\lambda controls the target probability mass assigned to a specified top quantile, providing an interpretable way to specify concentration independently of numerical reward values.

Critical weaknesses
  1. The rank-based transformation can substantially alter the original reward landscape. Although R∗R^\ast preserves score ordering, it discards differences in reward magnitudes: candidates separated by a large reward gap can receive the same relative weighting as candidates with nearly equal rewards, provided their ranks are unchanged. Consequently, the induced target p(x)∝R(x)p^(x)\propto R^(x) can differ substantially from the original target pR(x)∝R(x)p_R(x)\propto R(x).

  2. The theoretical analysis establishes ∣X∣|\mathcal X|-independent bounds for approximating the rank-based target distribution p∗p^\ast from reference samples. This is reasonable because the percentile function Φ\Phi, induced by the original reward RR, can be estimated without enumerating the terminal space. However, this does not eliminate the central difficulty of GFN learning, when high-reward candidates occupy an extremely small fraction qq of X\mathcal X. By Proposition 4, assigning a fixed, substantial probability mass to this region requires λ\lambda to scale approximately as 1/q1/q. The sufficient reference size in Theorem A.3 then grows quadratically with λ\lambda. Thus, independence from ∣X∣|\mathcal X| alone does not establish practical sample efficiency. Furthermore, accurately approximating p∗p^\ast through a reference-based reward does not imply that training a GFlowNet to sample from this target is easier or more sample-efficient than training one for the original reward-proportional distribution p(x)∝R(x)p(x)\propto R(x).

    This is demonstated in Table 4 and Tale 5. The fist part of approximating p∗p^\ast by p^\hat p works well, while the second part about PF⊤P_F^\top and p^\hat p can introduce substantial errors. It means λ\lambda specifies the desired concentration, but the trained GFlowNet does not always produce that concentration, and the chanllenge of GFN learning is not significantly alleviated by the new reward.

  3. The comparisons do not sufficiently isolate the advantage over comparably calibrated existing methods. RC-GFN’s λ\lambda is selected separately for each downstream task, with extensive sweeps in Appendix F. In contrast, the fixed temperatures and conditional temperature ranges of the baselines follow settings from previous work. This leaves it unclear how much of the discovery advantage comes from the proposed reward construction versus selecting a more suitable concentration.

  4. The diversity comparison does not control for differences in achieved concentration. In Figure 6, RC-GFN assigns approximately 63% of its probability mass to the top-10% set, whereas the tested OP-GFN and GR-GFN configurations assign nearly all their mass to that region. The observed diversity gap may therefore partly reflect different concentration levels rather than the benefit of the rank-based reward construction.

Questions for authors

Does RC-GFN achieve better discovery performance when each method receives comparable tuning? Does the rank-based reward help when a temperature baseline is calibrated to the same population top-set mass?

Which inverse temperatures were used to obtain the post-training results for Layer-GFN and Logit-GFN in Table 1?

Why is GR-GFN not reported in table 1 and Figure 3?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zmRT (Part 1/2)

We thank the reviewer for the careful reading and constructive feedback. Our main new evidence is a concentration experiment, in which the concentration parameter λ\lambda of RC-GFN and the inverse temperature β\beta of the temperature-controlled baselines are calibrated before training to the same promised mass on the top set TqT_q (Section 4.3 “CONCENTRATION EXPERIMENT” of the revised manuscript). We address your concerns point by point below.

Respond to Critical Weaknesses 1:

RC-GFN deliberately changes the reward-proportional target rather than attempting to approximate it. The loss of reward magnitude information is a consequence of this choice. In many design problems, scores serve as proxies for the desired outcome, and their numerical differences need not reflect reliable differences in utility, even when their ordering is informative. In such settings, it can be more useful to specify how much probability mass should be assigned to high-ranking candidates than to preserve the relative magnitudes of the scores.

RC-GFN provides this form of target specification. Under a uniform reference measure without score ties, Proposition 4 gives the population mass on the top-qq set as

p∗(Tq)=1−e−λq1−e−λ,p^*(T_q)=\frac{1-e^{-\lambda q}}{1-e^{-\lambda}},

independently of the reward scale and spacing. When reward magnitudes themselves carry meaningful preferences, a value-based target may be more appropriate. We have clarified this distinction in Section 3.3 "EXPONENTIAL RANK REWARD", and in Section 5 "DISCUSSION".

Respond to Critical Weaknesses 2:

Concern about the residual fitting error between PF⊤P_F^\top and p^\hat p

Following your comment on the fitting error between PF⊤P_F^\top and p^\hat p, we investigated the factors behind the large fitting errors reported in the original manuscript. We traced these errors to three factors. First, the policy network lacks the representational capacity to fit the target closely. Second, most terminating states in a training batch lie far below TqT_q, so without weighting they dominate the gradient, and little of it goes to the states near TqT_q that determine the promised mass. Third, the oracle budget is too small for the policy to converge. We therefore use a policy network with greater representational capacity, a reward weighting toward the top candidates and a larger oracle budget (Appendix H.4 “CONCENTRATION EXPERIMENT IMPLEMENTATION DETAILS”). With these changes, the fitting error falls on QM9, sEH and RNA, while the reference error stays small on all tasks (Appendix D.2 “CONCENTRATION EXPERIMENT”).

We use this setting in the new concentration experiment (Section 4.3). To keep the comparison fair, all methods in this experiment use the same larger policy network, the same flow matching objective and the same number of oracle calls, of which RC-GFN spends mm on its reference set. In this shared setting, RC-GFN still discovers more modes than the temperature-controlled baselines by the end of training on most tasks at P=0.9P=0.9, and it distributes its mass more uniformly within TqT_q.

TFBind10 remains the exception, where the fitting error stays large. At the same promised mass, the temperature-controlled baselines also place very little mass on TqT_q there, while RC-GFN places more. This is consistent with a policy-fitting difficulty shared across different target constructions on TFBind10, rather than a failure of the frozen-reference approximation alone. The analysis of TFBind10 is given in Appendix J “FAILURE CASE ANALYSIS”.

Concern about equating an accurate approximation of p∗p^* with easier training

We agree that an accurate approximation of p∗p^* does not by itself make the GFlowNet easier to train, and we have stated this explicitly in the section “DISCUSSION” of the revised manuscript. The contribution of RC-GFN lies in its target. First, under a uniform reference measure without score ties, the promised mass PP on TqT_q determines λ\lambda through the monotone equation in Proposition 4, independently of reward values. Second, the rank-based target weights depend on ranks rather than numerical score gaps. They are not uniform within TqT_q, but the learned RC-GFN policy has higher within-set uniformity in most settings of the concentration experiment above.

Concern about the large λ\lambda required at small qq

The reviewer is correct that concentrating a fixed, substantial probability mass on an increasingly small top quantile requires a larger λ\lambda. For a desired top-qq mass α\alpha, Proposition 4 implies

\qquad \text{as }q\to 0.$$ Thus $\lambda$ grows as $1/q$ in the extreme-tail regime. RC-GFN does not eliminate this cost. Its contribution is that $\lambda$ can be determined from the desired concentration without knowing the numerical spacing of reward values. We now state this scaling in Section 3.4 "SEMANTICS OF $\lambda$".
AuthorsOpenReview ↗

Response to Reviewer zmRT (Part 2/2)

Respond to Critical Weaknesses 2 (continue):

Concern about whether a ∣X∣|\mathcal X|-independent reference set implies sample efficiency

We agree that independence from the terminal-space size does not, by itself, establish sample efficiency. Theorem A.3 gives a general high-probability bound on the discrepancy between the frozen-reference target p^\hat p and the population target p∗p^*. At fixed accuracy and confidence, its sufficient reference size scales as O(λ2)O(\lambda^2).

Under a uniform reference measure without score ties, Proposition A.6 provides a sharper result,

\leq \frac{\lambda}{4m}.$$ This gives a sufficient reference size of $O(\lambda)$ for fixed root mean square error. Combined with $\lambda=O(1/q)$ at fixed desired top-set mass, the two bounds yield sufficient sizes of $O(1/q^2)$ and $O(1/q)$ under their respective assumptions. These results concern the accuracy of the reference-based target. They do not establish the oracle complexity of discovering high-reward candidates or the efficiency of fitting a GFlowNet policy to that target. We have made this distinction explicit in Section 3.5 "FINITE-REFERENCE ACCURACY" and in Section 5 "DISCUSSION". The new fixed-budget sEH ablation in Appendix E "CHOOSING THE REFERENCE SIZE" (Table 9) also shows the practical tradeoff: increasing $m$ from $30\lambda$ to $200\lambda$ reduces reference TV from 0.037 to 0.014 but lowers cumulative mode discovery from 1,082 to 739. **Respond to Critical Weaknesses 3 and Questions For Authors 1:** The reviewer is right that the mode discovery experiment alone cannot separate the effect of the reward construction from that of choosing a suitable concentration. The concentration experiment removes this difference, because $\lambda$ and $\beta$ are both solved before training from the same promised mass on the same top set $T_q$, and all methods use the same policy network, training objective and oracle budget. In this experiment, we compare the delivered mass, the uniformity of the policy within $T_q$ and the cumulative number of discovered modes (Section 4.3). Except on TFBind10, both RC-GFN and the baselines largely fulfill the promised masses. However, the policy of RC-GFN is more uniform within $T_q$, and RC-GFN discovers modes faster or discovers more modes on most tasks and promised masses. **Respond to Critical Weaknesses 4:** We agree that the diversity gap between RC-GFN and OP-GFN or GR-GFN on HyperGrid (Appendix C.3, Figure 7) partly reflects their different concentration levels. This difference is what the comparison examines. OP-GFN and GR-GFN have no parameter that sets their concentration, which is instead determined by the training length, so their top-set mass keeps rising towards one during training. Matching their concentration to that of RC-GFN would require stopping them at the step where they reach the same mass, which would give them a different oracle budget. We therefore exclude OP-GFN and GR-GFN from the concentration experiment above and make the matched comparison only with the temperature-controlled baselines, whose concentration can be set before training. **Respond to Questions For Authors 2:** Following Logit-GFN and Layer-GFN’s settings, each post-training sample draws its own $\beta$ from the exploration distribution used during training. We added this evaluation setting to Appendix H.2.2 “MODE DISCOVERY EXPERIMENT IMPLEMENTATION DETAILS”. **Respond to Questions For Authors 3:** GR-GFN was designed for multi-objective optimization, which differs substantially from our single-objective tasks, and its code is not publicly available, so faithfully reproducing its details on our downstream tasks was not straightforward, and we therefore reproduced GR-GFN in a minimal form and tested it only on HyperGrid. We have now added GR-GFN as a baseline to Section 4.2 “MODE DISCOVERY EXPERIMENT”, and its implementation details are given in Appendix H.1.2 “IMPLEMENTATION DETAILS”. RC-GFN discovers more modes than GR-GFN on sEH, TFBind10 and RNA, whereas GR-GFN discovers the most modes on QM9 (Figure 2, Table 1). To explain this, we computed the exact sampling mass that each trained policy places on the top set $T_q$ of each task, where $q$ is the quantile that defines the mode set. | Task | Target mass of RC-GFN | RC-GFN | GR-GFN | | --- | --- | --- | --- | | QM9 | 0.181 | 0.188 ± 0.017 | 0.454 ± 0.009 | | sEH | 0.139 | 0.267 ± 0.026 | 0.697 ± 0.038 | | TFBind10 | 0.131 | 0.034 ± 0.001 | 0.023 ± 0.000 | | RNA | 0.885 | 0.764 ± 0.008 | 0.182 ± 0.006 | GR-GFN has no parameter that sets its concentration, which instead keeps increasing with the training length. Within this budget, it places much more mass on $T_q$ than RC-GFN on QM9 and sEH. This higher concentration yields more modes on QM9 but fewer on sEH. On TFBind10 and RNA, GR-GFN places less mass on $T_q$ than RC-GFN and discovers fewer modes.

After go through the responses, I would like to maintain the reject recommandation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

Dear Area Chair and Reviewers,

Thank you for your constructive feedback. We have revised the manuscript to incorporate the additional experiments and clarifications discussed in our responses. The main revisions addressing the reviewers' concerns are summarized below:

  1. Concentration experiment. To compare concentration control under equal tuning budgets, the new Section 4.3 calibrates λ\lambda and β\beta to the same promised mass before training. RC-GFN spreads mass more uniformly within the top set and discovers more modes on most tasks.
  2. TFBind10 failure analysis. To examine the large fitting error on TFBind10, Appendix J tests various single-factor interventions. Comparing TFBind10 with the similar TFBind8 task, we find that the far more scattered modes of TFBind10 help explain this fitting error.
  3. Reference size, coverage and ties. To examine these factors under sparse rewards, we added a bounded Game of 24 experiment (Appendix K). A fixed-budget sEH ablation (Appendix E) shows that a larger reference improves accuracy but reduces mode discovery.
  4. GR-GFN baseline. We added GR-GFN to Section 4.2, where it discovers fewer modes than RC-GFN on sEH, TFBind10 and RNA. Its concentration grows with training length, which yields more modes on QM9 but overshoots on sEH.

We also clarified the scope of our theoretical claims in Sections 3.3 to 3.5 and 5 and Appendix A.4, and added the conditional baselines' post-training β\beta evaluation setting to Appendix H.2.2.

We would appreciate any further feedback on our revisions and responses. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.