acceptodds
Under review as a conference paper at ICLR 2027

J-Space-Guided Structural Search for Language Model Compression

Abstract

Structured pruning reduces language-model cost but risks removing computations needed across tasks. Capacity allocation alone leaves unresolved which equal-sized channel groups preserve useful semantic content. We propose candidate-dynamic J-space structural search to address this problem. The framework combines semantic mask characterization, joint capacity–identity optimization, and surrogate-assisted evaluation. First, a frozen Jacobian lens decomposes dense and pruned activations independently, exposing changes in sparse semantic support rather than imposing the dense model's support on every candidate. Second, a mixed-discrete representation combines component capacities and concrete masks under an exact parameter budget, while second-order compensation adjusts retained weights. Third, an online surrogate selects promising structures for full-model evaluation and updates a Pareto archive from measured behavior and semantic drift. Together, these components connect local semantic measurements to global structure selection without treating surrogate predictions as observed performance. On Qwen3.5-4B with 11.84% of parameters removed, two selected structures improve question-weighted accuracy by 1.10 and 1.19 percentage points over an equal-parameter non-J structural reference across eight benchmarks, with slightly lower WikiText-2 perplexity. The structures approach a conventional allocation reference, while results on CommonsenseQA, SciQ, and COPA reveal task-dependent trade-offs. These findings support semantic information as a useful structural-search signal, while leaving its advantage under fully matched search compute unresolved.

Primary area:
generative models
Keywords:
large language models, structured pruning, model compression, mechanistic interpretability, J-space, surrogate-assisted search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rxjkOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I think the paper studies the problem of pruning or compressing an already trained Transformer language model, although the paper never states the problem setup plainly. The method is based on "J-space search". Experiments are done on standard NLP benchmarks.

Critical strengths

None

Critical weaknesses

The paper is not written for humans to read. I can't rule out that an LLM would find it interesting or significant, but I have not inquired.

Questions for authors

I would like to see a human-written and human-understandable paragraph that explains the three components of the proposed method, as well as how they interact.

Also, what do “J-Space-Guided Structural Search” and “Candidate-Dynamic J-Space Search” mean in the two versions of the title?

Reviewer_SmzROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CJSS for structured LLM compression, jointly searching how much to prune and which channels to remove. J-space descriptions of activation changes guide a surrogate-assisted search, followed by optional recovery training.

Critical strengths

Searching mask identities at a fixed pruning budget is a reasonable direction. The local intervention and dictionary-control experiments provide some evidence that the J-space signal captures relevant changes. The authors also report negative results and acknowledge limitations openly.

Critical weaknesses
  • The main results give me little reason to choose CJSS. In Table 1, the conventional TyrAllocation reference achieves better accuracy and perplexity than both CJSS variants. Against ExpandedNonJ, the improvement depends on question-weighted averaging: macro accuracy is lower, and the separate public evaluation panel also favors ExpandedNonJ. The practical benefit looks quite limited even in the main, relatively mild compression setting.

  • The benefit of J-space in the full pipeline is still unclear. ExpandedNonJ was evaluated with a different search budget. Later experiments also change structure selection and recovery, which makes their improvements difficult to attribute. The local controls are useful, but the central comparison is still missing: J-on versus J-off with the same candidate space, search budget, and recovery schedule.

  • The added complexity has a weak practical payoff. The pipeline requires candidate-bank construction, semantic feature extraction, and surrogate search, while reported search costs exclude local-bank construction. The main results use hook-based models. The separately exported model reduces memory and improves time to first token, but decoding throughput barely changes. I am left unsure which deployment setting would justify the extra compression cost.

  • The paper needs a clearer takeaway. There are many experiments across local interventions, synthetic objectives, public benchmarks, and recovery, yet the connection between them remains loose. In particular, when does preserving the proposed semantic signal actually help downstream performance? The task-level trade-offs deserve an explanation. Figure readability also needs attention; it appears to me that Figure 1 is AIGC as there are a lot of rendering errors (blocks in (b) merged together, weird blocks on the left of the 'Transformer block' etc).

Overall, I recommend rejection. The approach is elaborate, but its central design has little demonstrated benefit over the conventional reference. The existing experiments leave both the contribution of J-space and the practical reason to use CJSS unresolved.

Questions for authors
  • Can the authors provide a controlled J-on/J-off comparison within the same search and recovery pipeline?

  • What is the total compression cost, including candidate-bank construction and J-space feature extraction, and where does CJSS offer a favorable quality–efficiency trade-off?

Reviewer_sM9mOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CJSS is a training-free structured width pruning framework that uses Jacobian J-space decomposition to guide mask selection in a search-based manner. CJSS remains training-free by using OBS to compensate for removed neurons in pruned layers. Additionally, a ridge-ensemble surrogate is used to update candidate scores within a Pareto archive for semantic drift and damage to update the set over which search is occurring.

Critical strengths
  • The papers formulation of using interpretabiliy J-Space tools for semantic decomposition of masks is an interesting and novel approach to pruning.
Critical weaknesses
  • Expanded NonJ is not well defined and outperforms both CJSS on most benchmarks. The paper briefly outlines it in Sec. 4.3 but provides no information on its search objective, construction, etc.
  • The results do not showcase that CJSS and J-Space information improves structure identification and the paper provides limited comparison to relevant baselines. These include other search-based structured pruning baselines [1], [2], [3], [4].
  • No definitions for the L_dep and L-light damage losses/scores.

References:

[1] Tang, Shengkun, et al. "Darwinlm: Evolutionary structured pruning of large language models." arXiv preprint arXiv:2502.07780 (2025).

[2] Malettira, Prajna G., et al. "TraceNAS: Zero-shot LLM Pruning via Gradient Trace Correlation." arXiv preprint arXiv:2602.02891 (2026).

[3] Sieberling, Oliver, et al. "Evopress: Accurate dynamic model compression via evolutionary search." arXiv preprint arXiv:2410.14649 (2024).

[4] Yuan, Tao, et al. "E 3^ 3-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models." arXiv preprint arXiv:2511.17205 (2025).

Questions for authors

Questions:

  1. Can you provide a matched ablation with the J-space turned off?
  2. What is the overhead of the per candidate J-Space decomposition?
  3. Does the J-space search extend to larger models and higher pruning ratios?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.