acceptodds
Under review as a conference paper at ICLR 2027

SWE-Quality: A Benchmark for Code Quality in Agentic Software Engineering

Abstract

As LLM agents increasingly automate software engineering tasks, a major bottleneck to deploying their code is **code quality**. Low-quality code is hard to read, hard to edit and expand, and may be less reusable. Repeated edits by agents over the lifetime of a repository may degrade its quality. Despite its importance, agentic benchmarks score correctness almost exclusively. We explore cheap-to-compute static metrics for evaluating agentic code quality. To that end, we introduce **SWE-Quality**, a benchmark built on DeepSWE that computes *quality scores* for agent-produced repository-level patches. SWE-Quality supplies **(i)** quality annotations for DeepSWE v1.1: 72 static metrics spanning a broad array of metric families, under 3 file-scope lenses (216 features), for 3,402 completions from 26 models, and the gold patches of its 33 Python tasks; **(ii)** a scoring protocol built for cross-model fairness, utilising within-task robust standardisation and tie-aware scoring; **(iii)** a validated scoring function, the *SWE-Quality Aligned Index*, a learned combination of those metrics that, on held-out tasks, reproduces 80% of the orderings a blind LLM judge (Claude Fable 5.1) gave to 1,204 within-task pairs of completions from 9 of DeepSWE's best performing models, and ranks those models almost exactly as the judge does (Spearman ); and **(iv)** a baseline leaderboard with ablations. By showing that cheap, deterministic and interpretable metrics reproduce most of an independent judge's quality orderings, SWE-Quality provides a scalable instrument for agentic code quality.

Primary area:
datasets and benchmarks
Keywords:
Benchmark, Code Quality, Agentic Software Engineering, Software Complexity Metrics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K1vVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SWE-Quality, a benchmark for evaluating the quality of code generated by software engineering agents beyond functional correctness. Built on DeepSWE v1.1, the benchmark covers 33 Python tasks and 3,402 patches generated by 26 models. The authors extract 72 static code metrics across three file-scope lenses, resulting in 216 features per patch, to characterize maintainability, extensibility, and understandability. To validate these metrics, the authors use an LLM judge to rank 290 functionally correct patches and train a learned scoring function, the SWE-Quality Aligned Index, using Extra Trees. The proposed index achieves approximately 80% agreement with the LLM judge on held-out tasks and a model-level ranking correlation of 0.95. The paper also includes comparisons with a second LLM judge, a small-scale human evaluation, and analyses of the relationship between code quality and functional correctness. The main contribution is a computationally efficient framework for evaluating code quality as an additional dimension of coding-agent performance.

Critical strengths

S1. An important evaluation dimension beyond functional correctness. The paper addresses an important limitation of existing coding-agent benchmarks, which primarily evaluate whether generated patches pass functional tests. Assessing maintainability, extensibility, and understandability is valuable, particularly as coding agents are increasingly used for repository-level software development. Separating code quality from correctness provides a potentially useful perspective on agent capabilities.

S2. A computationally efficient and reasonably systematic evaluation framework. The paper combines a broad collection of static code metrics with a learned scoring function and provides detailed descriptions of the metric definitions and evaluation protocol. The use of task-level cross-validation, multiple file-scope lenses, and comparisons with an independent LLM judge are useful methodological choices. The low computational cost also makes the approach potentially practical for large-scale coding-agent evaluation.

S3. interesting empirical insights into the relationship between code quality and correctness. The paper provides interesting empirical observations about the relationship between code quality and functional correctness across different coding agents. In particular, the results suggest that strong functional performance does not necessarily imply high code quality, and that reference implementations are not always preferred by the proposed quality evaluation. These findings provide useful insights into the limitations of correctness-based evaluation and further motivate the need for complementary code-quality metrics.

Critical weaknesses

W1. Limited validity of static code-quality metrics. The proposed benchmark relies on 72 static metrics to evaluate code quality, but it remains unclear whether these metrics can accurately capture maintainability, extensibility, and understandability in real-world software engineering. Metrics such as cognitive complexity, function length, and documentation coverage are only indirect proxies for code quality. The paper lacks sufficient evidence that improvements in these metrics correspond to better practical engineering outcomes.

W2. Insufficient reliability and representativeness of human evaluation. The human evaluation covers only eight tasks and involves four paper authors, with only one annotator per task. This setup does not allow for measuring inter-annotator agreement, and it is unclear whether the selected tasks and annotators are representative. Furthermore, the human annotators did not follow the same evaluation rubric as the LLM judge, raising concerns about the consistency and reliability of the evaluation.

W3. Insufficient consideration of repository-specific quality standards. Code quality is highly dependent on repository-specific characteristics, including architectural conventions, developer practices, and domain requirements. A patch with favorable static metrics may still violate the existing design principles or coding conventions of its repository. The proposed within-task normalization and file-scope lenses do not fully address these repository-specific differences, raising concerns about the fairness and generalizability of the resulting quality scores.

W4. Limited generalizability across programming languages and task types. The evaluation is restricted to 33 Python tasks, despite the broader scope of agentic software engineering. Different programming languages and software engineering domains may have substantially different quality requirements and coding conventions. It remains unclear whether the proposed metrics and learned scoring function can generalize beyond the evaluated Python repositories.

W5. Lack of validation against downstream maintenance outcomes. Although the paper motivates code quality through long-term maintainability and software evolution, its experiments focus on static code characteristics and agreement with LLM judgments. It does not examine whether patches receiving higher quality scores actually lead to easier subsequent modifications, fewer regressions, or lower maintenance costs. This limits the extent to which the proposed benchmark can be interpreted as measuring practical software maintainability.

Questions for authors

Q1. Human Annotation Reliability and Representativeness.

How were the eight tasks used for human evaluation selected, and how representative are they of the full benchmark in terms of repository characteristics, task difficulty, and implementation complexity? Why was each task evaluated by only one human rater? Without multiple independent annotations per task, how can the reliability of the human judgments be established?

Q2. Direct Validation of the Aligned Index.

The paper reports human-alignment results for individual static metrics and the unweighted Overall score, but not for the final Extra Trees-based Aligned Index. Could the authors directly evaluate the agreement between the learned index and human rankings, preferably using a task-held-out protocol to avoid potential calibration overlap?

Q3. Repository-Specific Quality Criteria.

How does the proposed index account for repository-specific development conventions, architectural constraints, and coding practices that may not be captured by static metrics? Could the authors investigate whether its effectiveness varies across repositories with different characteristics, or validate it on previously unseen repositories?

Q4. Sensitivity to Calibration Data and Model Selection.

How stable is the learned index when the selected passing completions or the models used for LLM-judge calibration change? In particular, could the authors evaluate whether the index systematically favors coding styles represented by the nine calibration models?

Q5. Cross-Language Generalizability.

Given that the main evaluation covers only Python tasks, what evidence supports the applicability of the proposed metrics and learned scoring function to other programming languages, where code structure, type systems, and development conventions may differ substantially?

3 comments

Thank you for a careful review. We have updated our draft to address your concerns.

W2 / Q1. Human evaluation

The human study is a pilot. We have further adjusted wording to reflect this fact, and no claim rests on it. A comprehensive human study would require a lot of resources and lead time, which we do not currently have, as each comparison requires significant effort from experienced developers. Looking at a very few samples, we can also estimate that human raters have much higher variance than the LLM judges, requiring many raters per task for a good estimate — again a massive undertaking.

Q2. The Aligned Index against human rankings

Due to the relatively high standard errors in the pilot human study (due to limited samples) we did not think it would be appropriate to fit a combined index, which requires additional training and validation splits. Furthermore, we no longer draw any conclusions from the pilot human study (see above).

W2. Judges from one family

We have since run two further complete rounds with judges from different families, each blind to authorship, each scoring all 290 passing completions on all 33 tasks.

Agreement over the 1,204 within-task ordered pairs. All three judges order 65.9% of pairs alike.

Judge pairPairs alikePooled SpearmanMean within-task τ
Fable 5.1 (Anthropic) – sol (OpenAI)79.6%+0.706+0.618
Fable – Gemini 3.8 Flash (Google)77.1%+0.689+0.566
sol – Gemini75.1%+0.606+0.533

Cross-family agreement is lower than the same-family figure. In our updated draft we use the median of the three judges' within-task robust z scores as the alignment target to mitigate any self-preference biases. The new index fitted to the median reaches concordance 0.778 ± 0.016 against 0.797 ± 0.018 for the single judge in the paper.

W3 / Q3. Repository-specific quality standards

Every feature is standardised within task, so each comparison is between patches to the same repository at the same commit, and a patch is never scored against a different codebase. That removes between-repository differences in absolute metric level, which is why we do not think the scores are unfair across repositories.

We agree with the real issue you bring up. A violation of a repository's architectural or stylistic conventions leaves no static trace. No feature in our set can see that, and we have updated our draft to state this limitation.

Q4. Sensitivity to the calibration pool

We held out each of the nine calibration models in turn, refit the index on the remaining eight, and scored the held-out model's own patches on held-out tasks. This needs no new judging, only a restriction on which rows the index may fit. The protocol is the paper's own — the same 80/20 task splits, 300 repeats, task-clustered standard errors — with the held-out model's rows dropped from the training fold, so the index has seen neither that style nor those tasks. Every figure below is paired against the full index on identical splits.

The index does not in general depend on having seen a model. Overall concordance moves by −0.002 on average, and the model ranking is close to fixed. The ordering of the remaining eight models has Spearman ρ 1.00 in eight of the nine refits and 0.95 in the ninth, where one adjacent pair swaps. For eight of the nine models the concordance of the pairs involving the held-out model is indistinguishable from the full index, between −0.008 and +0.005 with every z below 1.3 in magnitude.

One model is an exception. Gemini 3.7 Flash is the lowest-rated candidate by the judge. Holding it out costs 0.089 ± 0.021 concordance on its own pairs (z −4.3) and 0.017 ± 0.006 overall (z −2.7), and the aligned index becomes more generous to it, predicting −0.21 in place of −0.56 (with it in the calibration pool) against a true −1.35 (judge rating).

We purposefully chose strong models for the calibration pool, as we believe these are the models most useful for practitioners, and this demonstrates that the index is robust to the calibration pool when considering the top models.

W4 / Q5. Cross-language generalisation

The evaluation is 33 Python tasks and we have no cross-language evidence. We have adjusted wording and expressed this more explicitly in our draft.

Final remarks

Thank you again for your review. We hope that our response satisfies your questions and concerns. While we concede that we cannot directly attribute our metric to human-perceived quality (absent a sufficiently large human study), we demonstrate that it is a good proxy for powerful LLM judges, which are widely used in many benchmarks (due to feasibility), at a fraction of the cost. Furthermore, we acknowledge that this work cannot answer questions regarding long-term maintainability and software evolution. However, we believe we introduce generalizable tools that can be used for this purpose given a sufficiently large, open source, contamination free dataset with truly long horizon model completions, which does not currently exist and is extremely resource-intensive to create.

Thank you for the additional experiments and clarifications. However, my core concern remains unresolved. After stepping back from human alignment and downstream maintainability validation, the benchmark relies entirely on LLM-as-a-judge rubric scores as its ground truth. Yet rubric-based LLM scoring for code quality is itself noisy and biased toward surface coding styles (as seen in the 65.9% three-judge agreement). Fitting static metrics to an unvalidated LLM rubric proxy does not establish genuine software engineering quality. Therefore, I will maintain my original score.

Reviewer_4yMyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SWE-Quality, a way to evaluate code quality alongside functional correctness in coding-agent benchmarks. Using 33 Python tasks from DeepSWE v1.1, the authors measure 72 static code metrics for 3,402 patches from 26 models, as well as the maintainers' reference patches. They calculate these metrics over three different file scopes, giving 216 features in total. They then ask an LLM judge to rank passing solutions and use those rankings to train the Extra Trees-based SWE-Quality Aligned Index. On held-out tasks, the index matches 79.7% of the judge's within-task rankings and also comes close to reproducing the overall model ranking.

Critical strengths
  1. The paper addresses an important issue by considering both whether code is correct and whether it is well written.
  2. The paper goes beyond using stronger LLMs to assess code quality and explores whether more controllable static metrics can reproduce their judgments.
Critical weaknesses
  1. Both the main judge and the second judge are from Anthropic. The human study covers only eight tasks, with one rater per task, so it cannot measure agreement between human raters. The LLM judge's concordance with the human rankings is 65.2%, and the paper does not directly evaluate the final combined index against those human rankings.
  2. The average cognitive complexity metric alone reaches 76.6% concordance, compared with 79.7% for Extra Trees, a difference of 3.1 percentage points. The paper does not report a paired confidence interval for this difference or enough ablations, such as removing cognitive complexity. The model is fitted on only 33 tasks with 215 usable features, and repeating the same task splits does not add more independent tasks.
Questions for authors
  1. Could you evaluate the final combined index against independent human rankings, with several developers rating each task?
  2. What is the task-level paired confidence interval, or statistical significance, for the 3.1-point gain of Extra Trees over average cognitive complexity? Do the results still show a consistent gain if cognitive complexity is removed or if the model uses only a small number of the strongest metrics?
2 comments

Thank you. Both of your concerns are fair, and both are now addressed with new experiments that are included in our updated draft.

W1. Both judges were from Anthropic

Agreed. We have since run two further complete rounds with judges from different families, each blind to authorship, each scoring all 290 passing completions on all 33 tasks.

Agreement over the 1,204 within-task ordered pairs. All three judges order 65.9% of pairs alike.

Judge pairPairs alikePooled SpearmanMean within-task τ
Fable 5.1 (Anthropic) – sol (OpenAI)79.6%+0.706+0.618
Fable – Gemini 3.8 Flash (Google)77.1%+0.689+0.566
sol – Gemini75.1%+0.606+0.533

Cross-family agreement is lower than the same-family figure. In our updated draft we use the median of the three judges' within-task robust z scores as the alignment target to mitigate any self-preference biases. The new index fitted to the median reaches concordance 0.778 ± 0.016 against 0.797 ± 0.018 for the single judge in the paper.

Q1. The human study, and the index against human rankings

The human study is a pilot. We have further adjusted wording to reflect this fact, and no claim rests on it. A comprehensive human study would require a lot of resources and lead time, which we do not currently have, as each comparison requires significant effort from experienced developers. Looking at a very few samples, we can also estimate that human raters have much higher variance than the LLM judges, requiring many raters per task for a good estimate — again a massive undertaking.

W2 / Q2. The 3.1-point gain over cognitive complexity

Thank you for these insightful suggestions. We have now run the ablations on the paper's protocol (300 repeated 80/20 task splits, inner grouped CV for max_features, standard errors clustered by task), for paired tests and other concerns.

These are now run against the new median-of-three-judges target.

Feature setFeaturesConcordancePaired Δ vs best single
all (the paper's index)2150.778 ± 0.016+0.042 ± 0.020 (z +2.1)
no cognitive complexity2090.776 ± 0.017+0.039 ± 0.023 (z +1.7)
top 8 single features only80.771 ± 0.018+0.034 ± 0.019 (z +1.8)
no whole-repository lens1430.776 ± 0.017+0.039 ± 0.021 (z +1.8)
no size/volume features1470.761 ± 0.017+0.025 ± 0.016 (z +1.5)
size/volume features only680.759 ± 0.017+0.023 ± 0.023 (z +1.0)
avg_cognitive_complexity (union), unfitted10.737 ± 0.021—

The paired interval you asked for is the last column. It is taken per held-out task on the identical splits, with the standard error clustered by task, so it is the correct interval for the gain rather than the difference of two marginal numbers.

Direct answers to your questions:

The gain is real but small. The index beats the unfitted best single feature by +0.042 ± 0.020 of concordance, z +2.1. The CI excludes zero (|t| > 2, p < 0.05), and we have reported it this way.

Removing cognitive complexity costs 0.002 (0.776 against 0.778). Your concern that most of the signal comes from one metric does not hold up — the signal is distributed across the correlated complexity, nesting, duplication and documentation features, so no single one of them is load-bearing.

An index on the 8 best single features reaches 0.771. Therefore all 215 features are not required for good performance.

Final remarks

Thank you again for your insightful review. We believe it has significantly improved our draft. We hope that our response satisfies your concerns. While we concede that we cannot directly attribute our metric to human-perceived quality (absent a sufficiently large human study), we demonstrate that it is a good proxy for powerful LLM judges, which are widely used in many benchmarks (due to feasibility), at a fraction of the cost.

Thanks for the detailed responses. I think most of my concerns have been addressed. However, I still have some concerns about the reliability of the metric, especially since there is no reliable human validation or human evaluation combined with AI. So I would prefer to keep my original score, and I hope the authors understand my position. For the final acceptance decision, I will also take the opinions of the other reviewers and the AC into account.

Reviewer_rcJJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Agentic coding benchmarks evaluate correctness but not code quality. This paper studies whether cheap static metrics can approximate quality judgments. Using the 33 Python tasks of DeepSWE v1.1 (3,402 completions from 26 models, plus gold patches), the authors compute 72 standard metrics under three file-scope lenses, for 216 features in total. The reference signal is a single LLM judge (Claude Fable 5.1) that ranks passing completions from 9 models plus gold on each task. Average cognitive complexity alone agrees with the judge on 76.6% of held-out pairs, and an Extra Trees index over the features reaches 79.7%. On the resulting leaderboard, gold patches rank below 21 of the 26 models, and none of the metrics separates passing from failing completions.

Critical strengths
  • The evaluation protocol is careful. Only passing completions are judged, so correctness does not confound the comparison. Splits are held out by task, standard errors are clustered by task, and Table 12 reports all metric/lens combinations, including those that go against the expected direction.
  • The metrics are fully specified (App. B) and cheap to compute, so the results are easy to reproduce. The finding that a single unfitted metric agrees with a frontier judge on about 77% of pairs could be useful for designing quality rewards.
Critical weaknesses
  • All validation is against one Claude judge. The second judge is Claude Opus 5, from the same model family, so the 88.5% agreement mainly shows that the two judges have similar preferences, not that those preferences are correct.
  • The human study is limited: four raters who are all authors, one rater per task (so inter-rater agreement cannot be estimated), eight tasks, and a different candidate pool. From Table 8, I get a model-level Spearman correlation of about 0.40 between the human raters and the judge, and about 0.09 when gold is excluded.
  • Patch size is not controlled for, and the judge appears to prefer larger patches. halstead_bugs on the total lens is V/3000, i.e., a volume measure, and it orders about 65% of pairs in the same way as the judge (concordance 0.350, Table 12). There is no size-only baseline, and the size features are still included in the index.
  • The learned index improves on cognitive complexity alone by only 0.031 (Table 1), with no paired test, so most of the signal comes from one metric.
  • The benchmark does not add new tasks; it adds quality annotations to 33 existing DeepSWE tasks.
  • Minor: there are unfinished sentences and typos (L147, L196-197, L464, L473), and some standardisation details that matter for the results are only given in App. D.
Questions for authors

Clarifications (could affect my score)

  1. How well do size-only signals (e.g., lines added, total code_lines) agree with the judge? How does the index perform without the 69 context features? If it stays close to 0.80, my concern about size would be largely resolved.
  2. Could you add a judge from a different model family?
  3. Could you have several raters per task, so that inter-rater agreement can be measured? Were raters blind to which patch was gold? Can the index itself be evaluated against non-author raters?
  4. Who wrote the gold patches? L424 says the repository maintainers, but L259 says the tasks were written from scratch. Do the gold patches include tests or documentation that model patches lack? The finding that gold ranks low depends on this.
  5. How is a new model scored, given that the union lens and the standardisation both depend on the candidate pool?
  • Before the index is used as an RL reward, it would be good to test whether simple rewrites (splitting functions into trivial helpers, adding boilerplate docstrings) can game it.
2 comments

Thank you for the insightful review. We have updated our draft with the cross-family judge experiment and the size ablations you asked for.

W1 / Q2. A judge from a different model family

Your framing is correct. Fable agreeing with Opus at 88.5% shows the two share preferences, not that the preferences are right. We have since run two further complete rounds with judges from different families, each blind to authorship, each scoring all 290 passing completions on all 33 tasks.

Agreement over the 1,204 within-task ordered pairs. All three judges order 65.9% of pairs alike.

Judge pairPairs alikePooled SpearmanMean within-task τ
Fable 5.1 (Anthropic) – sol (OpenAI)79.6%+0.706+0.618
Fable – Gemini 3.8 Flash (Google)77.1%+0.689+0.566
sol – Gemini75.1%+0.606+0.533

Cross-family agreement is lower than the same-family figure, as you predicted. In our updated draft we use the median of the three judges' within-task robust z scores as the alignment target to mitigate any self-preference biases. The new index fitted to the median reaches concordance 0.778 ± 0.016 against 0.797 ± 0.018 for the single judge in the paper.

W3 / Q1. Patch size, and the context features

These are now run on the paper's protocol (300 repeated 80/20 task splits, inner grouped CV for max_features, standard errors clustered by task), against the new median-of-three-judges target. The last column is the paired difference against the unfitted best single feature, taken per held-out task on the identical splits, with the standard error clustered by task.

Feature setFeaturesConcordancePaired Δ vs best single
all (the paper's index)2150.778 ± 0.016+0.042 ± 0.020 (z +2.1)
no whole-repository lens1430.776 ± 0.017+0.039 ± 0.021 (z +1.8)
no cognitive complexity2090.776 ± 0.017+0.039 ± 0.023 (z +1.7)
top 8 single features only80.771 ± 0.018+0.034 ± 0.019 (z +1.8)
no size/volume features1470.761 ± 0.017+0.025 ± 0.016 (z +1.5)
size/volume features only680.759 ± 0.017+0.023 ± 0.023 (z +1.0)
avg_cognitive_complexity (union), unfitted10.737 ± 0.021—

A size-only index reaches 0.759, which is 0.019 below the full index and well above the best single metric. Size is sufficient but not necessary. Dropping every size feature costs only 0.017 (0.761), so the two blocks are largely redundant and the index is not simply a size measure. Additionally, the size signal could be confounded with completeness of contribution. As we note under Q4 below, 94% of model patches ship tests and 59% ship documentation, so a "larger" patch is usually one that also includes tests and docs.

On the bar you set, dropping the whole-repository lens costs 0.002 of concordance (0.776 against 0.778), so the index does not depend on the features computed outside the edited files.

W4. The margin over cognitive complexity

Agreed, and the paired test is the last column above. Against the median-consensus target the index beats the unfitted best single feature by +0.042 ± 0.020 of concordance (z +2.1), taken per held-out task on the same splits with the error clustered by task. The gain is real but small, and we have reported it this way instead of the bare 0.031 gap. Additionally, removing every cognitive-complexity feature costs 0.002 (0.776).

Q3. Human raters

The human study is a pilot. We have further adjusted wording to reflect this fact, and no claim rests on it. A comprehensive human study would require a lot of resources and lead time, which we do not currently have, as each comparison requires significant effort from experienced developers. Looking at a very few samples, we can also estimate that human raters have much higher variance than the LLM judges, requiring many raters per task for a good estimate — again a massive undertaking.

Q4. Who wrote the gold patches, and whether they contain tests or docs

The gold patches were written from scratch by software engineers working for Datacurve, and were never submitted to the projects (genuinely active OSS projects pinned at real commits). The wording is our error and has been fixed. Your instinct that the gold ranking depends on patch content is right, and the asymmetry runs opposite to the direction your question implies. Over the 290 judged completions, 94% of the 257 model patches touch a test file and 59% touch documentation, against 12% and 3% of the 33 gold patches. We have amended our draft to include these disclosures.

Q5. Scoring a new model

Both the union lens and the standardisation can be kept frozen at release. The union lens is a fixed per-task file list, computed once from the reference pool (containing all 26 models). The standardisation uses the per-task median and MAD of the reference pool, which we also publish. A new patch is standardised against those stored constants and never enters them.

A new model is therefore scored by running the analyser once to obtain the static metrics over the three lenses, applying the stored per-task constants and the frozen feature scaling, then evaluating the predictor (requires no refitting). The score is a deterministic function of that patch alone.

Final remarks

Thank you again for your detailed review. We believe it has made our submission stronger. We hope that our response satisfies your questions and concerns. While we concede that we cannot directly attribute our metric to human-perceived quality (absent a sufficiently large human study), we demonstrate that it is a good proxy for powerful LLM judges, which are widely used in many benchmarks (due to feasibility), at a fraction of the cost.

Comments on the paper

1 comment
Area_Chair_WvWYOpenReview ↗

Reminder from AC

Dear Reviewer,

As the ICLR discussion period is closing, please take a moment to read the authors’ responses carefully and let them know which of your concerns have been addressed and which questions remain open. If anything needs clarification or would benefit from further discussion, please raise it and engage with the authors and fellow reviewers while there’s still time.

Thank you for your time, your thoughtful reviews, and your service to the ICLR community.

Best Regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.