acceptodds
Under review as a conference paper at ICLR 2027

From Art to Engineering: An Architecture Evaluation Framework for Edge-Level Deep Graph Transformation

Abstract

Deep graph transformation lacks controlled evaluation protocols for comparing architectures across structurally different transformation tasks. We introduce a controlled evaluation framework for edge-level deep graph transformation that places ten architectures and thirteen graph-computation and real-world tasks under a shared evaluation contract, yielding sixteen directly comparable experiments. The framework is open-source, provides reproducible baselines for analysing architecture–task interactions, together with controls over graph topology, feature provision and supervision redaction, and welcomes community contributions of new tasks, architectures, and evaluation settings. Across it, no architecture is consistently strongest, and three ablation studies show that changing the information supplied to the learner can change relative architectural performance while the transformation itself remains fixed. Dense architectures, which do not use the graph’s edges to determine which nodes exchange information, can also match or exceed graph-native ones, leading in eight of the sixteen experiments. We define a task’s information regime as the information conditions under which it is learned: what task-relevant information is supplied, the structural range over which it must be resolved, and whether predictions depend on one another. We use cross-task analysis to motivate the information regime as a fifth factor that extends an existing task categorisation framework.

Primary area:
datasets and benchmarks
Keywords:
Deep Learning, Graph Machine Learning, Graph Neural Networks, Deep Graph Transformation, Graph-to-Graph Transformation, Architecture Evaluation, Open Source

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_t3xpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces an evaluation framework for edge-level deep graph transformation, comparing ten architectures across thirteen tasks and sixteen experimental settings. It investigates how architectural performance varies with feature availability, structural dependencies, and prediction constraints. Based on these observations, the authors propose an additional task-characterization dimension called the information regime.

Critical strengths

Motivation: There is some effort to establish a standardized evaluation framework for graph transformation. This is somewhat meaningful because existing methods are often evaluated under different task definitions and experimental settings.

Critical weaknesses

I sincerely feel that this work has not been carefully polished, as there are numerous formatting issues and a lack of rigorous presentation of the methodology and figures. For example, Table 1 even uses ellipses to omit experimental results, which I find rather inappropriate for a conference submission. The GitHub repository link is also redacted, making it difficult to directly inspect the claimed open-source framework, although supplementary code is mentioned.

More importantly, I am not convinced that this paper offers a substantial research contribution beyond collecting existing architectures and tasks into a unified evaluation framework. Most of the findings are rather expected, and the analysis does not provide sufficiently convincing insights into why particular architectures perform better under different settings. Given that the main contribution is supposed to be a rigorous and reproducible benchmarking framework, I would expect much greater attention to methodological clarity, experimental completeness, and overall presentation. In its current form, I do not think this work meets the standards expected of an ICLR submission.

Questions for authors

See weakness

4 comments
AuthorsOpenReview ↗

Response to Reviewer t3xp

Thank you for the review, and for recognising the motivation for a standardised evaluation framework for graph transformation. We address the specific points first and then the contribution.

Contributions

The review describes the paper as collecting existing architectures and tasks into a unified framework. We see each of the three parts differently:

  • Tasks. They were not existing benchmarks. The implementations are our own, and apart from Graph Denoising, which we include to compare against published results, none of the tasks had a neural baseline. There were no earlier results to collect.
  • Architectures. They are standard on purpose. A baseline is useful when it is a model the community already knows, so that a new graph transformation model can be judged against it.
  • Unified framework. As your review notes, existing methods are often evaluated under different task definitions and experimental settings. Here, every architecture is given the same task, supervision, features, and metrics, so differences between architectures can be separated from differences in setup. That is what makes the results usable as baselines.

Findings

The review finds most of the findings expected. An expectation needs earlier evidence, and without baselines there was none: we did not know before running the experiments which architectures would suit which task, or how they would respond to changes in the supplied information. Two outcomes surprised us the most, although many more were identified:

  • The graph structure is the output in every task, and dense architectures still lead in nine of the sixteen experiments. Graph Rewriting intuition led us to expect graph-native models to dominate, but that is not the case because of the specifics of message passing and, as the results show, the varying information regimes.
  • A target computed by a polynomial-time heuristic should be easier to learn than an NP-hard one. Estimated Minimum Chordal Completion is harder than the exact task for every architecture.

If prior work predicts these outcomes for graph-to-graph tasks, we would be grateful for references so we can cite them and compare against them.

On why architectures differ: where we could test an explanation, we did. The transformations are known, so we know which features carry much of the answer. In three experiments, we withhold those features and report how each architecture's score changes. We would welcome pointers to the explanations you find unconvincing, and to the evidence that would convince you, as this is the part we most want to improve.

Presentation and methodology

The review mentions numerous formatting issues. We would be grateful if you could point them out so we can fix them. The paper was proofread and line-edited by people outside the immediate specialism, and we eliminated any issues identified before submission.

Thank you for pointing out the incomplete methodology. The revision adds:

  • Per-task training configurations, threshold tuning, and checkpoint criteria (Appendix A.5, Tables 6 and 7).
  • A paragraph in Section 3.3 separating the shared evaluation conditions from architecture-specific training settings.

Please let us know if you find anything else missing.

Table 1

Table 1 is an abridged comparison of selected models with the published GOLD ablations, and the ellipsis marks the rows that continue in the full table. The full table, with all ten architectures and every metric, is Appendix Table 11 (Table 9 in the submitted version), and Table 1's caption points to it. The rows with the most important results were preserved, and the final table row was preserved to accurately show the scale of the results. This is a common convention in computer science papers.

Code

The complete anonymised repository was submitted as supplementary material, with per-run results and task seeds. The public GitHub URL is redacted because the repository identifies the authors, and the ICLR author guidelines explicitly allow code to be submitted as supplementary material.

Thank you for the response, however I don't think your rebuttal is sufficient enough to convince me this manuscript is publishable. For the formatting, i.e., I've never seen a paper use dotted line for experiment results. Authors reported a lot of experiment tables but yet most of them are lack of proper interpretation and analysis. I will keep my scores.

AuthorsOpenReview ↗

Response to Reviewer t3xp

Thank you for your follow-up and for considering our rebuttal. We appreciate your feedback, but respectfully disagree with your assessment. Our rebuttal addressed the specific concerns raised, and we explicitly requested clarification on the remaining issues. Your follow-up reiterates these concerns without identifying specific shortcomings in our explanations or revisions.

Presentation and reproducibility. We explained that the ellipses in Table 1 are a deliberate space-saving convention, abbreviating a comparison with published baselines. The two displayed leading rows show the best results for AUC and Recall@k, while the final row provides the lowest Recall@k as a reference point. The percentages permit direct comparison with the published GOLD baseline. All intermediate results appear in Appendix Table 11, which is explicitly referenced in the caption and accessible directly through a clickable link in the PDF. No experimental results are omitted from the paper. We also clarified that the complete anonymised source code, including task seeds and per-run results, was submitted as supplementary material. We asked you to identify the other formatting problems mentioned in your original review, but no additional examples were provided.

Methodology and analysis. We substantially expanded the methodological details in Section 3.3 and Appendix A.5, including training configurations, threshold selection, and checkpoint criteria. Sections 4.1–4.4 and Section 5 provide task-specific and cross-task interpretations of the experimental results. While your follow-up states that most tables lack proper analysis, it does not identify which results or explanations are insufficient. We explicitly asked for such examples and the evidence needed to strengthen our interpretations, but received no clarification.

Scientific contribution. Our rebuttal also explained why the work goes beyond collecting existing architectures and tasks: we implemented the tasks, established previously unavailable neural baselines under a controlled DGT setting, and developed a framework for systematically investigating architecture–task interactions. Your follow-up does not address this explanation or identify prior work establishing these contributions.

We respectfully ask you to reconsider your assessment in light of the revisions and explanations provided. If concerns remain decisive, we would appreciate specific examples of the deficiencies that our rebuttal has not addressed.

You clearly did not read through my comments. Please stop using GPT to generate more meaningless comments but rather try to find actual novelty and properly polish your manuscript . My view stands with other reviewers.

Reviewer_kBZgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper constructs an open-source evaluation benchmark for deep graph transformation (DGT) that includes 10 architectures and 13 tasks and focuses on structure learning. It spans feedforward, convolutional, attention-based, non-parametric, and graph-native models. The authors claim that no architecture is consistently strongest, while dense methods lead in 8 out of 16 experimental settings, outperforming graph-native ones.

Critical strengths
  1. The reviewer believes standardized evaluation frameworks are necessary and valuable, especially with such diverse baselines, including graph-aware and graph-free methods.
  2. The analysis of the informativeness of node features and the role of edge structure is interesting. The results support prior work in this domain.
Critical weaknesses
  1. Given the recent work in [1, 2, 3, 4, 5], the reviewer is unclear and skeptical about the contribution as well as the novelty of this paper. The works in [1, 2, 3] already derive similar conclusions to this work, calling for standardized benchmarks and arguing that most of the commonly benchmarked datasets in graph learning may not directly use the underlying edge structure, which hinders progress in the field. Accordingly, the authors of [4, 5] propose an extensive graph benchmark spanning graph-level, node-level, and edge-level tasks.
  2. The choice of tasks is arbitrary and datasets are mostly synthetic. The reviewer is concerned about the insufficient real-world evaluation, which is a key practical consideration.
  • [1] Sbicego, Luca, et al. "On the Role of Structure in Hierarchical Graph Neural Networks." I Can't Believe It's Not Better: Challenges in Applied Deep Learning. 2025.
  • [2] Errica, Federico, et al. "A Fair Comparison of Graph Neural Networks for Graph Classification." ICLR. 2020.
  • [3] Mesquita, Diego, et al. "Rethinking Pooling in Graph Neural Networks." NeurIPS. 2020.
  • [4] Bechler-Speicher, Maya, et al. "Position: Graph Learning Will Lose Relevance Due to Poor Benchmarks." ICML. 2025.
  • [5] Stoll, Timo, et al. "GraphNench: Next-Generation Graph Learning Benchmarking." arXiv preprint. 2025.
Questions for authors
  1. How does this paper distinguish itself from the existing literature mentioned above? Could you please elaborate on the main contribution of this benchmarking framework?
  2. How did you decide on the tasks and datasets? Could you incorporate more real-world structure-learning datasets to increase use-case value?
4 comments
AuthorsOpenReview ↗

Response to Reviewer kBZg

Thank you for the review, and for recognising the value of a standardised evaluation framework with diverse baselines and the analysis of feature and structural information. Thank you also for the pointers to [1–5].

1a. Relation to [1–5]

We think "similar conclusions" follows from a reading our text should have prevented, so we state the difference directly. None of [1–5] defines or evaluates a task as graph transformation, where a model maps an input graph to an output graph:

  • [1–3] concern graph classification, with and without pooling. The input is a graph, and the output is a single label. [2] is cited in the submitted version, and its protocol motivated ours.
  • [4] is a position paper on benchmarking practice in graph learning.
  • [5] benchmarks general graph learning tasks, while our framework specifically evaluates architectural suitability for learned graph-to-graph transformations under controlled information conditions.

We now discuss [4] and [5] in Related Work. The closest earlier benchmark is GraphGT, which covers more datasets and compares two transformation models (Interaction Networks and NEC-DGT).

As you summarise, [1–3] argue that commonly benchmarked datasets may not use edge structure. Our conclusion is a different one. In our tasks, the edge structure is what the model must output. Every architecture receives the same features which a task requests; in our experiments, these include features computed from the graph, such as adjacency powers, common neighbours, and shortest-path distances. Graph-native models additionally pass messages along the edges. We compare architectures on these shared inputs, and find that the best one depends on the transformation and on which information is supplied.

1b. Main contributions

  • Baselines where none existed. Apart from Graph Denoising, none of the thirteen tasks had a neural baseline before this work. We provide them for ten architectures under one protocol, so future models for these tasks have a point of comparison.
  • Comparable results. Earlier graph transformation models were each evaluated on their own data, features, and protocol. Our framework fixes the task, supervision, features, metrics, and model selection, so differences between architectures can be separated from setup differences. It is open-source, and new tasks can be added.
  • The information regime. Withholding the features that carry the answer changes the ordering of architectures in all three ablations, while an existing four-factor task categorisation places both conditions in the same category. The information regime captures what differs: the information supplied, the task's structural range, and the dependence between output predictions.

The revised Introduction also explains how the setting relates to link prediction, which is a special case.

We would also appreciate a clarification. The summary describes the paper as focusing on "structure learning". Could you tell us which line of work you have in mind? If it is methods that refine a graph to improve node classification, our setting differs, because the output graph is itself the training target. If it is learning to produce graph structure in general, then we agree, and we would be glad to hear which benchmarks you consider closest.

2. Task selection and real-world data

The selection follows a fixed criterion. We adopt the taxonomy of Machowczyk and Heckel [6], which defines sixteen tasks spanning four dimensions: complexity, input–output relation, graph type, and required structural change. We include all thirteen that fit a fixed-vertex edge classification output, as Section 3.2 states. The other three need node prediction or edge regression.

Nine tasks are generated, and this is deliberate: every graph has an exact target, and graph size and structure are controlled. Four tasks use public real-world data: SNAP Facebook Combined, SNAP Higgs Twitter, PowerGraph, and ConceptNet-N20 (following GOLD).

We agree that more real-world tasks would add value, and the framework supports adding them. A task needs an input graph and a target graph on the same nodes. If you know of public data in that form beyond what we use, we would welcome the suggestion and would like to include it.

The revision also adds per-task training configurations and model-selection criteria (Appendix A.5).

[6] Machowczyk and Heckel. Benchmark first: Defining tasks for graph transformation learning. In Graph Transformation, LNCS 16624, pp. 223–234. Springer, 2026.

Reviewer_kBZgOpenReview ↗

Response to rebuttal by authors

Thank you for the response.

Indeed, I meant learning to produce a graph structure in general. This directly relates to link prediction, which is covered by some of the papers I mentioned. I could not manage to find the revision in the introduction regarding this. Please highlight your revisions in a color other than black. Moreover, GraphGT is already very adjacent to this work. From my current understanding, the difference is the additional tasks and datasets, which unfortunately remain incremental.

At this current state, I would like to keep my score as is.

AuthorsOpenReview ↗

Response to Reviewer kBZg

Thank you for your follow-up and for clarifying what you mean by structure learning. We appreciate your engagement and would like to address the remaining points.

Link prediction and manuscript revisions. We agree that link prediction is related to DGT, and the revised Introduction explicitly acknowledges it as a special case (Section 1, lines 42–45). We have also expanded Related Work where relevant. However, the cited studies investigate related graph-learning problems, and their conclusions do not establish architectural suitability across graph-to-graph transformations. Regarding locating revisions, the conference provides reviewers with a PDF diff through the revisions panel.

Relationship to GraphGT and novelty. We agree that GraphGT is related, as already acknowledged in Section 2. However, the statement that our contribution consists of "additional tasks and datasets" mischaracterises both works. GraphGT's stated objective is to address the scarcity and limited diversity of graph generation and transformation datasets, providing a large dataset collection, unified data interfaces, and benchmark results. Its graph transformation experiments compare two architectures across seventeen datasets. Our objective is to establish an architecture evaluation framework for DGT, enabling systematic comparisons of ten architectures under shared task definitions, feature provision, supervision, and evaluation protocols. We implemented graph transformation tasks, established previously unavailable neural baselines across algorithmic neural reasoning and real-world tasks, and used controlled experiments to investigate architecture–task interactions and motivate the information regime. GraphGT already covers more datasets; expanding dataset coverage is not our contribution. The two works address different research objectives through substantially different experimental methodologies, as explained in our rebuttal and revised Section 2.

We respectfully ask you to reconsider your assessment based on the actual contributions and revisions. If you still consider the work incremental, we would appreciate clarification of which specific contributions are already established by GraphGT or the other cited studies.

Thank you for the response. Similar to the other reviewers, I also believe that the manuscript currently lacks the necessary presentation quality and in-depth analysis to be publishable. I will maintain my score.

Reviewer_4GDyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces a unified evaluation framework for edge-level deep graph transformation, benchmarking ten architectures across thirteen tasks. Through controlled feature ablations and cross-task analysis, the authors investigate architecture–task suitability and propose information regime as an additional dimension for task characterization.

Critical strengths
  1. Unified evaluation framework. The paper provides a useful framework for systematically comparing diverse architectures across graph transformation tasks.

  2. Broad empirical coverage. The experiments cover diverse graph structures and tasks, with feature ablations highlighting architecture-dependent performance differences.

Critical weaknesses

W1. Limited novelty and scientific insights. Prior work has already investigated architecture–task suitability, feature availability, and the competitiveness of non-graph models. The proposed information regime largely reorganizes these known observations without establishing substantially new insights or predictive principles for architecture selection.

W2. Insufficiently controlled comparisons. Graph-native architectures retain structural features (e.g., common neighbors) in their edge decoders even when corresponding input features are ablated. Some of these features directly encode task labels, making it difficult to disentangle architectural advantages from differences in available information.

W3. Limitations in task formulation and evaluation. Several graph-level optimization tasks are reduced to independent edge classification with a single target solution, despite potentially having multiple valid outputs. Consequently, classification metrics may not faithfully reflect the architectures' ability to solve the underlying graph transformation problems.

Questions for authors

Please address each of the weaknesses mentioned above.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 4GDy (1/2): W2

Thank you for the detailed review, and for recognising the value of the unified evaluation framework and the breadth of the empirical study. W2 identified a real problem, which we have corrected, so we address it first.

W2. Controlled comparisons

You are right, and we appreciate you raising it. In the submitted version, the graph-native edge decoder always added its own common neighbour, Jaccard, and Adamic–Adar features, independently of what a task requested. The appendix noted this for the two affected tasks, but it amounted to label leakage: in Shortcut Elimination these features encoded the label, and in the Transitive Closure ablation they gave graph-native models two-hop information that the other architectures lacked.

We removed the mechanism entirely. Every architecture now receives exactly the features a task requests (Appendix A.3). Shortcut Elimination requests no feature derived from the length-two path. The Transitive Closure ablation withholds A²–A⁵ and keeps Jaccard and Adamic–Adar for all ten architectures (Appendix C.1). We re-ran the three affected experiments, Shortcut Elimination and both Transitive Closure variants, for all ten architectures:

ResultSubmittedRevised
Shortcut Elimination, graph-native F1≈0.9990.780–0.801
Shortcut Elimination, leaderGIN, GPSCNN (0.835)
Transitive Closure, full-feature leaderRF (0.8895)Patch Transformer (0.8751), RF 0.006 behind
Transitive Closure ablation, F1 lost by RF59.7%33.7%
Transitive Closure ablation, F1 lost by Patch Transformer13.5%4.6%
Experiments led by a dense architecture8 of 169 of 16

In Shortcut Elimination, the graph-native models lose the leaked feature, which lowers their scores. In the Transitive Closure ablation, the dense models now receive Jaccard and Adamic–Adar as well, which raises their scores. The other thirteen experiments are unchanged: their feature requests were adjusted so that every architecture receives exactly the inputs it received before. Both main conclusions hold: no architecture is consistently strongest, and rankings change with the information supplied.

AuthorsOpenReview ↗

Response to Reviewer 4GDy (2/2): W1 and W3

W1. Novelty and insight

Apart from Graph Denoising, these tasks had no neural baselines, so we did not know before running the experiments which architectures would suit which task, or how they would respond to changes in the supplied information. The main results:

  • Transitive Closure: Random Forest is within 0.006 F1 of the lead with A²–A⁵ supplied and loses 33.7% without them. The Patch Transformer loses 4.6%.
  • Symmetric Closure: with the reverse edge supplied, dense models score 0.986–1.0. Without it, no architecture exceeds 0.73.
  • Molecular Bond Inference: reducing the atom descriptors moves the lead from GPS to CNN.
  • Dense architectures lead in nine of the sixteen experiments. Graph Rewriting intuition led us to expect graph-native models to dominate, which is not the case due to the specifics of message passing and, as the results show, the varying information regimes.
  • Estimated Minimum Chordal Completion, whose target is a polynomial-time heuristic, is harder than the exact NP-hard task for every architecture.

The ablations are possible because the transformation is known, so we know which features carry the answer and can withhold exactly those. The studies we know that relate performance to available information (Liu et al., 2022; Mao et al., 2024) analyse existing datasets, where this is unknown. The revised Related Work states the difference.

The competitiveness of non-graph models is known for node and graph classification, and we cite it (Huang et al., 2021). We are not aware of it being known in graph-to-graph transformation, where the graph structure is itself the output.

The review notes that prior work has already investigated these questions. We would be grateful for pointers to the studies you have in mind, in particular any that vary the supplied information for a fixed graph-to-graph task, so that we can compare against them directly.

We agree on one point: the information regime describes our results and has yet to be validated as a predictor of the best architecture on unseen transformations. We present it as an empirical characterisation.

W3. Non-unique targets

We agree. For Spanning Tree, Feedback Edge Set, and the two chordal tasks, each graph is labelled with one valid solution, so per-edge agreement can penalise a different valid output. The Conclusion and Appendix C.2 say so, and we read those results within that limit. For Spanning Tree we tried arborescence targets, golden trees, structure-based objectives, and output distributions; all performed worse than the single-tree target, which we did not expect (Appendix C.2). Where the objective is graph-level, we already score at graph level: Algebraic Connectivity is evaluated by the gain in the Laplacian eigenvalue.

We have yet to measure graph-level validity for the other constrained tasks, and the revision names this as future work. Would the share of valid outputs (a spanning tree, an acyclic graph, a chordal graph) be the measure you have in mind? We would welcome your view on what to report.

Thank you for the response. The authors have adequately addressed W2 by correcting the feature leakage and rerunning the affected experiments.

However, my concerns regarding W1 and W3 remain unresolved. Considering these issues, along with the concerns about presentation quality and insufficient analysis raised by other reviewers, I do not believe the paper currently meets the standards expected for publication at ICLR. I encourage the authors to further refine the work in future submissions.

I will maintain my original score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Request for Reevaluation

Dear Area Chair and Reviewers,

Thank you for the time and care devoted to reviewing our work. We have taken the feedback seriously and substantially revised the manuscript. The discussion has improved the framework's methodological reliability and reproducibility, and helped us clarify its contribution, experimental scope, and limitations.

Methodological correction. Reviewer 4GDy identified an unintended inconsistency in the features supplied to graph-native edge decoders. We acknowledged and removed the mechanism, reran all ten architectures in the three affected experiments, and updated the results and analysis. We particularly appreciate this observation: it resulted in a concrete improvement to the framework. Reviewer 4GDy subsequently confirmed that W2 had been adequately addressed.

Contribution and relation to prior work. A recurring concern was how our work differs from existing graph-learning studies and GraphGT. We clarified this in the revised Introduction and Related Work. GraphGT principally addresses the availability and diversity of graph generation and transformation datasets, alongside common processing and evaluation infrastructure. Our purpose is to establish a controlled architecture evaluation framework for edge-level DGT: ten architectures are compared across thirteen tasks and sixteen experimental settings under shared task specifications, feature provision, supervision, and evaluation protocols. We implemented the selected transformations and established neural baselines under this controlled DGT setting for twelve tasks that previously lacked them. Our feature interventions allow us to study changes in architectural suitability while holding the transformation fixed, motivating the information regime as an empirical characterisation. The revision also explains how link prediction relates to the broader graph-to-graph setting. We do not present the information regime as a validated predictor for unseen tasks.

Task selection and evaluation scope. Task selection follows a pre-existing taxonomy and a fixed inclusion criterion, including four tasks evaluated on real-world datasets. Our common edge-level metrics measure agreement with specified transformation targets. Some optimisation tasks admit multiple valid outputs, and we now explicitly discuss the limits of per-edge evaluation and complementary graph-level measures as future work. These qualifications clarify how our results should be interpreted; they do not change the supervised comparison that the framework is designed to support.

Clarity, analysis, and reproducibility. We expanded the reporting of task- and architecture-specific training configurations, model selection, thresholds, and methodology. The revised paper contains task-level and cross-task interpretations of the results. The abbreviated Table 1 points to the complete graph-denoising results in Appendix Table 11; the anonymised source code, task seeds, and per-run results were submitted as supplementary material.

Throughout the discussion, we tried to address every concrete concern and asked for specific examples, references, or additional evidence wherever criticisms were unclear. The follow-up responses maintained their assessments, but did not identify new implementation flaws or specific incorrect results following the corrections. We recognise that scientific significance and the appropriate scope of a benchmark involve judgement, and we welcome constructive disagreement. Our aim has consistently been to improve the work and provide an engineering foundation for reproducible DGT research.

We respectfully ask the reviewers and Area Chair to reconsider their assessments in light of the revised manuscript and detailed responses. If anything remains unclear or a particular claim seems unsupported, please do not hesitate to raise it. We would genuinely welcome the opportunity to clarify or address it.

Thank you again for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.