acceptodds
Under review as a conference paper at ICLR 2027

SHarP: Saliency-based Pruning of Agent Harnesses

Abstract

Agent harnesses are systems that coordinate model calls, tool use, and task execution to help large language models complete complex tasks. To meet task requirements and address failures, these systems are often iteratively refined by amending and patching their instructions, tools, and workflows, continuously increasing harness complexity. It is therefore unclear whether some resulting harness modules are redundant, introducing substantial token overhead with little, if any, performance gain. Inspired by neural network pruning, in this paper, we study harness pruning as a means of striking a better balance between task performance and token cost. We propose SHarP (Saliency-based Harness Pruning), a simple yet effective pruning strategy based on the saliency of each harness module with respect to performance and efficiency. Specifically, we first identify tools, instructions, and supporting mechanisms as components that can be individually disabled. We then estimate the saliency of each module by ablating it and assessing its task performance and token cost relative to the full set of single-module ablations. Modules with the smallest contribution to performance or largest computational overhead are subsequently pruned. Our evaluation across various harnesses on held-out validation sets reveals a surprising finding: most harnesses that we studied are highly redundant and can maintain comparable performance and efficiency even after a substantial portion of their modules are pruned. Our pruning approach and empirical findings provide new perspectives on agent harness design and optimization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agent harnesses, harness pruning, harness optimization, LLM agents

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_fZ7BOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper focuses on identifying the redundancies in harness modules and presents a framework for pruning them to minimize tokens while preserving performance. The components to be pruned are identified by ablating them individually to observe the impact on token cost and performance. Through their experiments over many harnesses, they find that many existing harnesses have redundant components.

Critical strengths

S1: The problem is very relevant as development efforts quickly lead to bloated features in the agentic harnesses

S2: The broad evaluation and qualitative analysis act as very useful datapoints for future works on harness optimization

Critical weaknesses

W1: My fundamental concern is that the work conflates harness redundancy with performance on a specific evaluation dataset. Harness components may serve particular functional purposes like clarification, error recovery etc. that may be marked as redundant if the evaluation dataset does not include inputs that trigger these. Pruning solely on the benchmark performance may compromise capability and usage in niche settings the harness may be designed for. Often, the theoretical reasoning for a component can be more reliable than an imperfect and incomplete benchmark at evaluating its utility.

W2: Single-module ablations do not capture interactions between components. E.g. Component B is only effective once component A triggers, but if B is marked as being impactful on performance before A is pruned, and A is later identified as a redundant component, then pruning it makes B redundant but it will not be pruned OR two modules with overlapping functionality do not show performance difference with single ablations, but removing both impacts performance. I do not see how these type of complex interactions can be captured with the current setup

W3: The performance efficiency tradeoff is difficult to control and developers can only set the number of modules to be removed, not the desired token reduction or acceptable performance degradation.

W4: Previous works (Beyond the Model: Demystifying Harness Effects in Software Engineering Agents by Hu et al. 2026) show that harness effectiveness depends on model backbone. Since all experiments are run on only one model, this creates a question how effective the approach is across different models?

Questions for authors

Q1: I would appreciate a discussion on how W1 is tackled by the current approach.

Q2: How does harness pruning change for different model backbones from small to large models?

5 comments
AuthorsOpenReview ↗

Response to Reviewer fZ7B -1

We thank reviewer fZ7B for your valuable comments.

W1/Q1. Can benchmark-based pruning mistake rarely triggered functions, such as clarification or error recovery, for redundancy? How does SHarP account for their functional importance?

SHarP targets pruning for a specific task distribution or application scenario. A component’s functional purpose explains when it may help, but its practical value also depends on how often it is needed, what other mechanisms remain available, and its recurring cost. A rarely needed component can therefore present a meaningful performance–cost trade-off. Our evaluation examines these trade-offs within the target scenario; its conclusions do not imply that a pruned component is unnecessary in every setting.

We also tested whether functional judgments would identify better pruning configurations on retail. We asked an LLM to review all modules and identify those requiring protection based on task requirements and module descriptions, without access to ablation results. It identified two such modules. One of the two modules also narrowly misses SHarP’s original performance-protection threshold (p = 0.0515), showing that its potential importance is already reflected in the saliency results. We retained these alongside SHarP’s original 13 protected modules, then evaluated the resulting pruning path on the validation set. The table shows the non-dominated configurations from each path, using median token cost and success rate.

Pruning pathModules retainedSuccess rateMedian tokens
Original SHarP2275.0%83.5k
Original SHarP1982.5%84.5k
Original SHarP4387.5%110.2k
SHarP with two additional protected modules2767.5%84.2k
SHarP with two additional protected modules1577.5%88.2k
SHarP with two additional protected modules3382.5%88.6k

Every configuration on the additional-protection path is matched or exceeded in success rate by an original SHarP configuration that uses fewer median tokens. For example, original SHarP reaches 82.5% success with 19 modules and 84.5k tokens, compared with 33 modules and 88.6k tokens after adding protection. Thus, the additional functional judgments do not yield a better observed trade-off.

AuthorsOpenReview ↗

Response to Reviewer fZ7B -2

W2. Single-module ablations miss dependencies and overlapping functionality that can change module importance during pruning. How does SHarP address these interactions?

We agree that dependencies and overlapping functionality can change a module’s importance after other modules are removed. SHarP uses single-module ablations to estimate importance at a limited evaluation cost. Although these estimates do not explicitly model interactions, they may still provide useful guidance for pruning. We conducted an additional random-drop experiment to examine whether modules identified as important by SHarP remain relatively important when multiple modules are removed together.

We sampled 60 configurations on LIFE-harness retail, independently retaining each module with probability 0.7, and evaluated each on the development set. For each module, we compared success rates and mean token costs between its enabled and disabled groups using one-sided Welch tests. Because other modules vary within both groups, these comparisons measure contributions across different module combinations, including interactions present in those combinations. We ranked modules separately for each metric in ascending order of p-value, with smaller p-values indicating stronger evidence that retention improves success rate or reduces token cost.

For token efficiency, the seven modules originally identified as important have a mean random-drop rank of 16.9, versus 25.8 for the remaining modules. Five of seven remain in the top half, suggesting that SHarP's efficiency estimates retain useful information for pruning. The table shows all seven modules, with rank 1 indicating the strongest evidence for retention.

ModuleOriginal efficiency rankRandom-drop efficiency rank
Exchange item existence check128
Order action guidance22
Multi-item exchange skill37
Category-scope skill412
Item multiplicity check523
Variant price extrema642
Confirmed-action skill74

For success rate, only 1 of the 48 modules shows a significant benefit from retention under random drop (p ≤ 0.05). The 13 originally protected modules have a mean rank of 26.9, near the middle of the ranking. However, the largely inconclusive random-drop results do not rule out the performance risks identified by the original screening. In efficiency-oriented pruning, SHarP takes a conservative approach by retaining these modules to reduce those risks, even if this gives up some potential token saving.

Although random drop allows module importance to be assessed across different combinations of other modules, it requires repeated sampling to populate each module’s enabled and disabled groups. Obtaining sufficient observations in both groups for every module usually requires more configurations than the number of modules. SHarP instead requires only one single-module ablation per module, giving it a fixed screening budget determined by the number of modules. Overall, this simple screening can still provide useful pruning guidance in an efficient way.

W3. How can users choose a pruning configuration for a desired token reduction and acceptable performance loss, rather than only specifying a module count?

We thank the reviewer for suggesting more direct control over token savings and performance loss. SHarP determines a pruning path. Although our original procedure does not explicitly take a desired token reduction or acceptable performance loss as inputs, these preferences can guide where to stop along that path. We illustrate this on LIFE-Harness retail using the original protected set and saliency order. We follow the pruning order on the development set, removing three modules at a time. When the token target is reached, we test removing only the first one or two modules from the last group to see whether less pruning would suffice. We choose the stopping point and configuration using only the development results, then evaluate the selected configuration on the held-out validation set.

With a 10% mean-token reduction target and a 15 percentage-point performance-loss tolerance, this selects the 37-module configuration. In single-run validation on held-out tasks, it reduces mean token cost by 13.7%, while success rate increases from 67.5% to 80.0%. A 20% target yields no qualifying configuration along the path evaluated on the development set. This illustrates how user preferences can guide the choice of pruning degree within SHarP’s existing pipeline.

AuthorsOpenReview ↗

Response to Reviewer fZ7B -3

W4/Q2. How effective is SHarP across model backbones, and how does pruning change from smaller to larger models?

We thank the reviewer for suggesting experiments across model sizes. We have added the appendix subsection “A.7 Harness pruning across model sizes,” marked with a blue title, containing detailed experimental results and discussion. The experiments run the full SHarP pipeline separately with Qwen3.5-35B-A3B and Qwen3.5-397B-A17B on LIFE-harness retail and JIT agentfold shopping, with one validation run per configuration. We summarize the main observations below.

Pruning provides larger benefits on 35B in these experiments, with configurations that use fewer tokens and achieve higher observed performance on both benchmarks. On 397B, shopping still offers substantial token savings at some performance cost, while retail offers limited savings. One possible explanation is that stronger models can use existing harness components more effectively, whereas weaker models benefit more from simplifying a harness whose components they use less effectively. The contrast between retail and shopping on 397B also shows that pruning opportunities depend on the task and harness.

Reviewer_NiQhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SHarP studies whether agent harnesses contain modules that can be removed without losing much task performance. It manually divides each harness into switchable tools, instructions, and support mechanisms. Single-module ablations on development tasks produce performance and token-cost scores, which are used to protect some modules and order the removal of the rest. The resulting configurations are evaluated on separate validation tasks. The study covers four harnesses in five settings and finds useful reductions in token use, while also showing that removing workflow support can increase cost or cause failure.

Critical strengths
  1. The paper asks a useful question that is easy to overlook when improving an agent by adding more instructions and mechanisms. The held-out curves show that some much smaller harnesses remain competitive. For example, Table 18 reports 35.0% success for OpenHands with three candidate modules retained, versus 29.4% for the full harness, with lower mean token use.

  2. The study covers different kinds of harness modules and tasks, rather than pruning only agents or communication links. The appendix carefully explains what remains available after each removal. This is especially important for interpreting the all-pruned results, which do not mean that every harness has lost all tools or task instructions.

  3. The negative results are informative. PaperQA2 and JIT show that removing modules can increase token use, and the trajectory examples explain how loss of stopping rules or evidence filtering causes this. Reporting these cases makes the paper more useful than a list of successful reductions alone.

Critical weaknesses
  1. The experiments show that some harnesses can be simplified, but not whether SHarP chooses better modules to remove than simpler rules. Table 1 compares two SHarP variants; the remaining controls are the full and all-pruned harnesses. Random pruning orders and ordering by mean token savings are not tested. Without these comparisons, the benefit of the proposed saliency rule remains unclear.

  2. The rule for protecting modules needs more justification. Equations (1)-(5) compare each removal with the median of all single-module removals, not with the full harness. A removal can be close to that median and still reduce performance relative to the full harness. The appendix explains how the test is calculated, but does not show whether using the full harness as the reference would change the protected modules or the final results.

  3. The paper does not define what counts as comparable performance. For PaperQA2, Table 17 shows a drop from 61.40% to 56.14% while mean token use falls by 11.8%. This may be a reasonable trade-off, but the paper does not say whether that loss is acceptable. The ranges across three trials in Figure 2 also do not measure uncertainty from the sampled tasks, so the strength of the performance-preservation claim is hard to judge.

Questions for authors
  1. Does SHarP outperform random pruning orders or ordering by mean token savings? Please compare them using the same protected modules, tasks, and pruning budgets.

  2. Why use the median across single-module removals as the reference for protecting modules, rather than the full harness? How do the protected modules and final results change if the full harness is used instead?

  3. How much performance loss is acceptable when claiming comparable performance? For the highlighted configurations, can you report confidence intervals for the score change from the full harness, comparing the results on the same tasks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer NiQh -1

We thank reviewer NiQh for your valuable comments.

W1/Q1. Does SHarP offer an advantage over random pruning and mean-token ordering under the same protected sets, tasks, and pruning budgets?

Following your suggestions, we evaluated random pruning as a baseline and mean-token ordering as an ablation of SHarP’s saliency rule on LIFE-harness retail and JIT shopping. We first present the random-pruning comparison. We generated two permutations of the unprotected modules using different random seeds and cumulatively removed modules along each order. All three paths retain the same protected set and are evaluated at the same retained module counts on the same validation tasks. We use one rollout per task and configuration for all paths and shared baselines.

Following SHarP’s pruning pipeline, we removed modules in groups of three along each ordering, yielding 11 intermediate configurations on retail and 5 on shopping. We evaluated all of these configurations, along with the shared baselines. The tables show all SHarP Pareto configurations and random-order configurations on the joint Pareto frontier. A configuration is on this frontier if none of the three paths offers higher performance at the same or lower token cost, or lower token cost at the same or higher performance.

Retail

Method or baselineRetainedPerformanceMean tokens (k)Joint Pareto
Full4867.50%121.203No
SHarP2577.50%100.315Yes
SHarP3780.00%104.611Yes
SHarP1982.50%107.013No
SHarP4387.50%117.835Yes
Random 11685.00%106.945Yes
All-pruned085.00%100.516—

Shopping

Method or baselineRetainedPerformanceMean tokens (k)Joint Pareto
Full2381.77%333.790Yes
SHarP1471.71%183.621Yes
SHarP2076.94%198.322Yes
All-pruned056.44%662.524—

SHarP configurations dominate 21 of the 22 intermediate random-order configurations on retail and all 10 on shopping. Only one random-order configuration remains on the retail joint Pareto frontier, and none remains on shopping. These results show that SHarP consistently finds better performance–token trade-offs than random pruning across the evaluated configurations.

As an ablation, we replaced efficiency saliency with mean-token ordering, prioritizing removals with the lowest mean token cost under the same protected sets, tasks, and pruning budgets. The two orderings select identical configurations at 4 of 11 pruning levels on retail and 4 of 5 on shopping. This agreement is expected when token savings are consistent across tasks, since both orderings prioritize removals that reliably reduce cost. Both also contribute distinct configurations to their joint Pareto frontier on retail, so neither fully dominates the other.

However, SHarP’s saliency accounts for task-level variability, reducing the influence of long-tail token costs on pruning priorities. For example, one retail task used 643.6k tokens when bulk_order_completion was disabled, versus a median of 106.6k across tasks. Removing this task from all ablations shifts the module’s priority from 1st to 12th under mean-token ordering, but only from 10th to 11th under SHarP, among the same 35 unprotected modules. This robustness benefit requires no additional task executions, since both orderings reuse the same ablation results.

AuthorsOpenReview ↗

Response to Reviewer NiQh -2

W2/Q2. Why use a median reference rather than the full harness to protect modules? How does this choice affect the protected set and final results?

We chose the median reference to identify modules worth retaining at a limited evaluation cost. Using the full harness as the reference makes these decisions depend on its observed task outcomes, which can vary substantially across runs. Reducing this dependence requires repeated full-harness evaluations, so we instead construct a per-task median reference from the single-module ablations already collected.

For comparison, we added an experiment using the full harness as the reference. We recomputed the protected sets using each of three full-harness runs and their per-task average, keeping the ablation data and one-sided paired t-tests unchanged. The table reports the number of protected modules at p ≤ .05.

ReferenceRetailShopping
Full run 102
Full run 255
Full run 391
Full, three-run average62
Median135

Changing only the full-harness run substantially changes the protected set, illustrating the sensitivity to a single reference evaluation. Every module protected using the three-run full reference is also protected using the median reference in both benchmarks. This more conservative choice reflects our aim to retain modules that may be important for task performance, even if doing so gives up some potential efficiency gains.

We also evaluated the pruning paths using the protected sets obtained with the three-run full reference, while ordering the remaining modules by their original efficiency saliency. Representative single-run validation results are shown below.

BenchmarkProtection referenceRetained modulesPerformance (%)Mean tokens (10³)
RetailMedian4387.50117.8
RetailFull, three-run average1887.50100.9
ShoppingMedian2076.94198.3
ShoppingFull, three-run average875.62170.2

In retail, the full-reference configuration shown achieves the same observed performance with fewer tokens. In shopping, it offers lower token cost at slightly lower observed performance than the median-reference configuration. These examples show that a smaller protected set can allow additional efficiency gains. The median reference instead prioritizes retaining potentially important modules while avoiding the extra full-harness evaluations needed to construct the reference.

W3/Q3. What performance loss is acceptable when claiming comparable performance? Please report paired confidence intervals for score changes from the full harness to quantify uncertainty across tasks.

The acceptable performance loss depends on the application’s priorities. Users willing to sacrifice some performance for lower token cost may prefer different configurations from those who cannot tolerate any performance drop. Analyzing trade-offs and identifying Pareto-optimal configurations therefore helps users choose a pruning level suited to their needs. We have revised the abstract and introduction to describe these trade-offs more precisely, avoiding a general claim of performance preservation.

We have added paired 95% confidence intervals and two-sided paired t-test p-values for performance differences from the full harness to all validation configuration tables in the appendix, with additions marked in blue. Comparisons pair the same validation tasks, averaging repeated trials within each task. Representative results are shown below.

BenchmarkConfigP (%)T (10³)ΔP95% CI for ΔPp-value
LIFE retailFull (48)75.00119.1———
LIFE retailPruned (43)82.50113.6+7.50[−0.68, +15.68]0.071
LIFE airlineFull (44)83.33138.7———
LIFE airlinePruned (27)83.33116.50.00[−10.12, +10.12]1.000
JIT shoppingFull (23)76.10329.1———
JIT shoppingPruned (14)75.96192.0−0.15[−5.80, +5.51]0.958
PaperQA2Full (18)61.401256.6———
PaperQA2Pruned (15)56.141108.4−5.26[−13.91, +3.39]0.225

These results illustrate why the preferred pruning level depends on the application. Retail, airline, and shopping offer token savings with higher, unchanged, or nearly unchanged observed performance, while PaperQA2 presents a clearer trade-off between lower cost and lower observed performance. The confidence intervals help users assess the uncertainty in these differences when choosing among configurations on the Pareto frontier.

Dear Reviewer NiQh,

Thank you again for the constructive review. We have posted detailed responses to all the points you raised, and would be glad to address any further questions. As the discussion period is coming to an end, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns. We sincerely appreciate your time and consideration.

Thank you for the detailed response and additional experiments. The comparisons with random pruning and mean-token ordering, the analysis of alternative protection references, and the paired confidence intervals substantially address the empirical gaps raised in my review. Some uncertainty remains about the advantage over mean-token ordering and the justification for the median reference, particularly given the limited repeated evaluation. Nevertheless, the new evidence strengthens the paper’s practical contribution, and I am raising my rating from weak rejection to weak acceptance. Please incorporate these comparisons and clearly state the remaining limitations in the revised manuscript.

Reviewer_TSqVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of pruning agent harnesses to reduce unnecessary complexity and inference cost while maintaining downstream task performance. The authors propose SHARP (Saliency-based Harness Pruning), a pruning framework inspired by neural network saliency analysis. SHARP estimates the importance of harness modules — including tools, prompts, workflow guidance, memory mechanisms, and execution constraints — through single-module ablations on a development set. The framework introduces two notions of saliency: performance saliency, measuring how much removing a module hurts task success, and efficiency saliency, measuring whether removing a module increases token cost. Based on these estimates, SHARP performs either efficiency-oriented or performance-oriented pruning and evaluates the resulting harnesses on held-out benchmarks. Experiments across different harnesses show that many existing harnesses contain substantial redundancy, and that moderate pruning can often reduce token cost while preserving or even improving observed performance.

Critical strengths
  1. The paper studies an important and underexplored problem. While recent work mainly focuses on expanding agent capabilities, this paper instead asks whether existing harnesses are unnecessarily over-engineered, which is both practically relevant and timely.

  2. The qualitative trajectory analyses are insightful. The examples showing how certain prompts induce unnecessary loops, or how removing completion tools can increase overall cost, help clarify why harness pruning is fundamentally different from standard neural pruning.

  3. The writing is clear and easy to follow.

Critical weaknesses
  1. The methodology relies heavily on single-module ablations, which may not adequately capture higher-order interactions between harness components. The paper itself presents several examples where the effect of removing one module depends strongly on whether other modules are present, suggesting that saliency estimates may not generalize reliably across pruning configurations.

  2. The paper focuses primarily on token cost rather than end-to-end runtime or deployment efficiency. In practice, latency, external tool overhead, and execution stability may matter as much as token usage, especially for tool-using agents.

  3. Some conclusions occasionally feel overstated. The experiments demonstrate that moderate pruning can sometimes improve efficiency and even observed task performance, but this does not necessarily imply that current harnesses are systematically over-engineered in a general sense.

Questions for authors

Please refer to Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer TSqV -1

We thank reviewer TSqV for your valuable comments.

W1. Can saliency from single-module ablations reliably guide pruning when modules interact?

We agree that interactions can change a module’s importance when other modules are removed. SHarP uses single-module ablations to estimate importance at a limited evaluation cost. Although these estimates do not explicitly model interactions, they may still provide useful guidance for pruning. We conducted an additional random-drop experiment to examine whether modules identified as important by SHarP remain relatively important when multiple modules are removed together.

We sampled 60 configurations on LIFE-harness retail, independently retaining each module with probability 0.7, and evaluated each on the development set. For each module, we compared success rates and mean token costs between its enabled and disabled groups using one-sided Welch tests. Because other modules vary within both groups, these comparisons measure contributions across different module combinations, including interactions present in those combinations. We ranked modules separately for each metric in ascending order of p-value, with smaller p-values indicating stronger evidence that retention improves success rate or reduces token cost.

For token efficiency, the seven modules originally identified as important have a mean random-drop rank of 16.9, versus 25.8 for the remaining modules. Five of seven remain in the top half, suggesting that SHarP's efficiency estimates retain useful information for pruning. The table shows all seven modules, with rank 1 indicating the strongest evidence for retention.

ModuleOriginal efficiency rankRandom-drop efficiency rank
Exchange item existence check128
Order action guidance22
Multi-item exchange skill37
Category-scope skill412
Item multiplicity check523
Variant price extrema642
Confirmed-action skill74

For success rate, only 1 of the 48 modules shows a significant benefit from retention under random drop (p ≤ 0.05). The 13 originally protected modules have a mean rank of 26.9, near the middle of the ranking. However, the largely inconclusive random-drop results do not rule out the performance risks identified by the original screening. In efficiency-oriented pruning, SHarP takes a conservative approach by retaining these modules to reduce those risks, even if this gives up some potential token saving.

Although random drop allows module importance to be assessed across different combinations of other modules, it requires repeated sampling to populate each module’s enabled and disabled groups. Obtaining sufficient observations in both groups for every module usually requires more configurations than the number of modules. SHarP instead requires only one single-module ablation per module, giving it a fixed screening budget determined by the number of modules. Overall, this simple screening can still provide useful pruning guidance in an efficient way.

AuthorsOpenReview ↗

Response to Reviewer TSqV -2

W2. Metrics beyond token cost, such as latency, external tool overhead, and execution stability, also matter in practice. How does SHarP address these metrics?

We agree that practical deployment also depends on other metrics such as latency, external tool overhead, and execution stability. We focused on token cost because it is less sensitive than latency to serving load, network conditions, and tool response times, allowing more reproducible comparisons across runs. However, SHarP is generalizable to different metrics. To demonstrate this, we added a latency-oriented pruning experiment using measured task duration as the efficiency metric in the same screening and pruning procedure. We reused timing records from the single-module ablations to compute latency saliency and evaluated the resulting configurations on LIFE-Harness retail and JIT shopping validation tasks, with one run per task. For each setting, the table shows the configuration with the lowest observed mean validation latency along the pruning path, alongside the full and all-pruned baselines.

SettingModules retainedPerformanceMean latencyLatency change vs full
LIFE retail48 (full)75.0%114.6 s—
LIFE retail3775.0%91.2 s−20.5%
LIFE retail0 (all-pruned)77.5%104.5 s−8.9%
JIT shopping23 (full)81.8%811 s—
JIT shopping2073.5%343 s−57.8%
JIT shopping0 (all-pruned)56.4%1,629 s+100.8%

Performance denotes success rate for retail and match rate for shopping. The selected SHarP configurations significantly reduce latency relative to full under one-sided paired t-tests (p = .005 and p < .001, respectively). Retail shows a latency reduction at the same observed success rate, while shopping illustrates a larger reduction with a performance trade-off. These results demonstrate that SHarP can guide pruning according to the efficiency metric relevant to the application.

W3. The observed pruning gains do not justify a general claim that harnesses are over-engineered.

Thank you for pointing out the overly broad wording. We have revised the abstract and introduction to limit our conclusions to the evaluated harnesses and task distributions, and to clarify that the performance–efficiency trade-offs vary across settings. These changes are marked in blue in the revised manuscript.

Dear Reviewer TSqV,

Thank you again for the constructive review. We have posted detailed responses to all the points you raised, and would be glad to address any further questions. As the discussion period is coming to an end, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns. We sincerely appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.