- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors propose TopoPlanner, a framework designed to lift tool-dependency graphs into two-dimensional cellular workflow complexes, aiming to address task planning challenges faced by large language model (LLM) agents when dealing with non-linear workflows. By leveraging cosheaf-consistency-based subcomplex retrieval and multidimensional structural reasoning, the paper extracts topology-aware graph tokens to guide the model in generating tool sequences. The authors evaluate their framework on an augmented benchmark dataset containing non-linear structures, demonstrating performance improvements across key metrics.
Critical strengths
Introducing high-order topological tools into the domain of LLM agent task planning offers a novel research perspective.
Demonstrating good empirical results on datasets containing non-linear structures such as loops, branches, and merges.
Critical weaknesses
The mathematical formulation appears unrigorous to me: Equation (8) merely performs pairwise cosine alignment rather than establishing true categorical cosheaf consistency. In Equation (9), because the retrieval objective relies on a simple arithmetic mean without size or connectivity constraints, how do the authors prevent the algorithm from severely degrading into selecting a single high-scoring node? Could the authors also provide the missing discrete optimization pseudo-code used to solve this retrieval objective?
I am concerned about the stability of cellular lifting, as it heavily depends on the arbitrary choice of the underlying spanning tree. How sensitive is the framework when different spanning trees fundamentally alter 2-cell neighborhoods and message-passing paths? Additionally, how do the authors justify using an unoriented cycle topology, given its evident semantic misalignment with actual directed control-flow (such as DAG merges versus feedback loops)?
The ablation studies fail to isolate the true causal contribution of the topological representation. Have the authors considered evaluating against control baselines with randomized 2-cell associations or equal-capacity hypergraph models to prove that the gains stem from correct topology rather than added parameter capacity?
I found the evaluation protocol quite opaque regarding how flat text sequences generated by LLMs are parsed back into directed graphs with loops and merges. Without reporting standard deviations across multiple runs, how can we rule out overfitting to synthetic data motifs? Moreover, why is quantitative evaluation of computational complexity and inference latency missing, given that introducing high-order cellular retrieval and multidimensional reasoning undeniably increases overhead?
I question the practical utility of relying so excessively on pre-constructed static topologies in real-world open environments. How does the framework maintain robustness when facing dynamic API schema updates, parameter changes, or temporary service disruptions? Does the framework offer any online incremental update mechanism, and what is the computational cost of rebuilding these cellular complexes from scratch?
Questions for authors
please refer to Weaknesses
5 comments
2nd Response to Reviewer 3Xfc
W2. Does the spanning forest change the representation, and what does an unoriented face mean?
For a fixed spanning forest of the underlying undirected graph, each non-tree edge closes one fundamental cycle. The unfiltered cycles form a basis of the same cycle space under different forest choices. Our implemented length bound retains only short cycles; the retained 2-cells and incidence neighborhoods can therefore change with the forest. In a fixed-checkpoint test that changes the prepared face incidence consumed by the model, the default forest yielded 54 short faces on HF and 85 on MM; ten alternate forests yielded 42–62 and 53–77.
Table X2. Fixed-checkpoint forest sensitivity, N-F1 / L-F1 (%). Alternative values are mean ± sample SD across ten forests.
| Domain | Inference budget | Default forest | Default repeat | Ten alternative forests |
|---|---|---|---|---|
| HuggingFace | Original | 81.83 / 55.95 | 81.74 / 55.76 | 81.94±0.09 / 56.07±0.17 |
| Multimedia | Original | 88.32 / 68.30 | 88.37 / 68.26 | 88.38±0.10 / 68.37±0.16 |
| HuggingFace | Half-budget stress test | 82.33 / 56.69 | 82.29 / 56.71 | 81.09±0.70 / 54.93±0.94 |
| Multimedia | Half-budget stress test | 87.96 / 67.85 | 88.06 / 68.01 | 87.90±0.14 / 67.75±0.17 |
Under the original budget, aggregate scores vary little across the tested forests, while the more selective HF half-budget diagnostic shows that stability depends on retrieval budget. The half-budget rows change the inference policy and are not additional training-seed runs. A DAG merge may form a cycle in its undirected view without being directed feedback. The 2-cell marks that joint region; directed 1-cells retain prerequisite direction. This keeps the representation result distinct from observation-conditioned loop execution.
W3. Is the gain due to correct regional incidence rather than extra capacity?
The reason for introducing 2-cells is not that directed edges cannot represent prerequisites. Rather, a merge or loop-shaped workflow couples several such edges through a shared boundary; pairwise propagation leaves that joint region implicit. TopoPlanner uses the boundary incidence of each 2-cell both to exchange information across the region and to retrieve the region together with its boundary. This is the distinction the component study is intended to test.
Table X3 shows the full model, the edge-only 1-skeleton, and a hypergraph alternative across both backbones and domains. On Multimedia, Vicuna-7B reaches 66.24 L-F1 with the full model, compared with 50.13 without 2-cells and 59.73 with hypergraph grouping; CodeLlama-7B reaches 69.59, versus 56.35 and 62.79. The same ordering holds on HuggingFace. Removing boundary closure or the consistency term also lowers performance in every configuration. Taken together, these comparisons connect the gain to how regional boundaries are represented and used, rather than merely to the presence of an additional high-order unit.
Table X3. Component diagnostics on HuggingFace and Multimedia; N-F1 / L-F1 / Acc (%).
| Backbone | Variant | HuggingFace N/L/Acc | Multimedia N/L/Acc |
|---|---|---|---|
| Vicuna-7B | w/o 2-cell | 75.58/46.61/30.88 | 73.29/50.13/34.15 |
| Vicuna-7B | w/o retrieval | 76.07/47.88/32.21 | 77.97/55.11/35.37 |
| Vicuna-7B | w/o consistency loss | 80.69/53.31/38.68 | 81.47/58.29/40.98 |
| Vicuna-7B | no Eq. 9 consistency | 77.39/49.09/33.68 | 80.95/57.03/41.71 |
| Vicuna-7B | no boundary closure | 75.66/47.07/32.50 | 81.41/57.71/43.17 |
| Vicuna-7B | hypergraph variant | 79.47/51.66/37.65 | 82.38/59.73/44.63 |
| Vicuna-7B | TopoPlanner | 82.23/55.72/42.35 | 86.80/66.24/54.76 |
| CodeLlama-7B | w/o 2-cell | 78.95/51.87/35.29 | 75.90/56.35/43.54 |
| CodeLlama-7B | w/o retrieval | 79.75/56.96/35.59 | 83.89/62.92/49.27 |
| CodeLlama-7B | w/o consistency loss | 80.80/53.83/38.82 | 87.12/67.61/53.54 |
| CodeLlama-7B | no Eq. 9 consistency | 79.44/53.19/38.24 | 86.24/64.34/53.41 |
| CodeLlama-7B | no boundary closure | 78.74/52.00/35.59 | 85.59/66.14/55.37 |
| CodeLlama-7B | hypergraph variant | 77.35/50.24/31.32 | 84.20/62.79/49.15 |
| CodeLlama-7B | TopoPlanner | 83.43/59.62/47.35 | 89.08/69.59/62.20 |
We will make this mechanism and the different ablation settings clearer in the revision: the edge-only and hypergraph variants probe the representation, while the closure and consistency rows probe how the retrieved regions are used. The evidence supports the boundary-aware design; it does not require claiming that no other formalism can encode the same workflow.
3rd Response to Reviewer 3Xfc
W4. What is recovered from a flat sequence, and what do variance and runtime measurements show?
TopoPlanner introduces topology before decoding: retrieved, boundary-closed regions inform which tools the LLM selects and how it orders their dependencies. Its final output remains an ordered tool list, as in the benchmark's common planning interface. We assess the resulting nonlinear-structure predictions directly. Across four domains, merge/reuse edge recall rises from 19.04% with GTool to 39.72% with TopoPlanner on Vicuna-7B (Table X4; 804 examples and 2,325 relevant gold edges per backbone). On Multimedia, the stricter example-level dependency-order score rises from 8.29% to 40.37% (Table X5); missing prerequisites count as errors. These gains show that structural conditioning improves the selection and ordering of tools involved in shared dependencies, despite the common sequence output.
Table X4. Merge/reuse dependency prediction (%).
| Backbone | Method | Examples | N-F1 | L-F1 | Merge/reuse edge recall |
|---|---|---|---|---|---|
| Vicuna-7B | GTool | 804 | 51.18 | 22.06 | 19.04 |
| Vicuna-7B | TopoPlanner | 804 | 76.16 | 46.92 | 39.72 |
| CodeLlama-7B | GTool | 804 | 64.56 | 35.27 | 29.06 |
| CodeLlama-7B | TopoPlanner | 804 | 78.94 | 49.91 | 41.01 |
Table X5. Gold-dependency ordering, Dep. / Pair (%).
| Backbone | Method | HF | MM | Daily Life |
|---|---|---|---|---|
| Vicuna-7B | GTool | 9.71 / 24.18 | 8.29 / 23.69 | 39.86 / 57.32 |
| Vicuna-7B | TopoPlanner | 27.94 / 51.74 | 40.37 / 64.29 | 60.14 / 79.50 |
| CodeLlama-7B | GTool | 25.74 / 44.91 | 32.56 / 49.73 | 48.11 / 62.30 |
| CodeLlama-7B | TopoPlanner | 33.24 / 58.64 | 48.78 / 71.80 | 63.57 / 83.28 |
The benchmark's shared scorer projects adjacent predicted tool pairs onto the available directed tool graph to compute L-F1. We will explain this mapping and define Acc precisely as the fraction of examples with node F1 at least 0.99, so neither measure is mistaken for tool-execution success. The reported 10,000-resample paired bootstrap measures uncertainty over matched test examples for fixed models; we will label it accordingly and distinguish it from variation across training runs.
We also quantify the cost of this structural interface. Closed-subcomplex retrieval takes 22.89–39.45 ms per request on CPU across the four evaluated graphs (Table X6). In same-card, reverse-order H800 profiles, Full adds 5.36–24.35 ms per request relative to Hypergraph in the measured static-inference comparisons (Table X7).
Table X6. Retrieval-pool size and CPU retrieval mean (50 fixed queries).
| Domain | Nodes / directed edges / retained 2-cells | Edge candidates | CPU ms |
|---|---|---|---|
| HuggingFace | 23 / 225 / 54 | 225 | 23.65 |
| Multimedia | 40 / 449 / 85 | 449 | 33.11 |
| Daily Life | 40 / 1,560 / 146 | 512 | 39.45 |
| ToolBench | 245 / 366 / 19 | 366 | 22.89 |
Table X7. Static CodeLlama-7B model inference, batch 1, ten warmup batches excluded (ms/request).
| Domain | Same-card order | Full mean / median / p95 | Hypergraph mean / median / p95 | Full − Hyper mean |
|---|---|---|---|---|
| HuggingFace | Full → Hypergraph | 540.47 / 512.83 / 822.60 | 525.63 / 507.36 / 774.44 | +14.84 |
| HuggingFace | Hypergraph → Full | 532.27 / 510.73 / 787.72 | 507.92 / 488.19 / 750.43 | +24.35 |
| Multimedia | Full → Hypergraph | 622.07 / 601.50 / 925.23 | 616.71 / 595.68 / 931.90 | +5.36 |
| Multimedia | Hypergraph → Full | 618.47 / 599.59 / 923.50 | 605.68 / 586.00 / 914.46 | +12.79 |
These measurements characterize the planning stages; downstream tool calls are outside their timed scope.
Response to Reviewer 3Xfc
We thank the reviewer for recognizing the value of representing higher-order workflow dependencies and for pressing on the mathematical formulation, lifting stability, structural controls, evaluation protocol, and open-environment cost. Our central claim is specific: a 2-cell makes a region jointly formed by valid tool dependencies available to request-conditioned retrieval and finite-depth structural reasoning. A directed graph can encode the same dependencies; an unoriented region is not itself an executable feedback policy. We answer the five weaknesses in their original order and distinguish structural plan prediction from runtime task completion throughout.
W1. What do Eqs. 8–9 and the retrieval algorithm actually establish?
Eq. 8 supplies the incidence-level compatibility signal used in cellular retrieval. In its displayed form, each coface-to-boundary term is nonnegative and, for nonzero vectors,
1 − cos(a,b) = ½||â − b̂||². Zero residual on every incidence of a selected, boundary-closed subcomplex therefore gives agreement of the normalized representations along its included boundaries. This is useful specifically for the proposed regional representation: selecting a 2-cell also retains the edges and nodes against which that region is assessed. The result concerns compatibility of the selected representations. Whether learned incidence maps satisfy path-composition identities is a separate, stronger question that Eq. 8 alone does not test. We will state the local zero-residual result with its assumptions and distinguish it from categorical cosheaf consistency.Eq. 9 describes the relevance–compatibility trade-off; its mean-score form alone does not impose a minimum subcomplex size. For example, a node scoring 1 while all other cells score lower is a feasible singleton optimum. The implemented selector instead uses an explicit node budget followed by closure-aware edge/face selection. The existing appendix algorithm describes cycle-basis construction; to make retrieval reproducible, we will add the following selection steps:
Table X1 shows the resulting selections under the evaluated default budgets. The non-singleton behavior follows the budgeted algorithm; it is not inferred from the unconstrained mean objective or from an exact PCST guarantee.
Table X1. Default-budget retrieval trace; mean edge/face counts.
The component diagnostic reports lower structural scores when consistency or boundary closure is removed, supporting their operational role in the planner. We will make the mathematical and implementation descriptions consistent: the inspected implementation uses a gated squared incidence residual, while the displayed Eq. 8 uses cosine. The revision will state the appropriate local compatibility result and the actual budgeted retrieval procedure without treating either as a proof of functoriality.