acceptodds
Under review as a conference paper at ICLR 2027

Pruning-Based Reinforcement Learning for Compression Operator Composition with Adjustable Objective Weights

Abstract

Compression algorithm selection depends on application preferences over com- pression rate, compression time, and decompression time. We therefore use an adjustable weighted objective combining compression rate, compression time, and decompression time. Compression algorithms share operators that either reduce data size or transform data to facilitate compression. We identify and recombine these operators to seek a lower objective value than fixed operator pipelines. We propose operator-composition reinforcement learning (OCRL). It combines group relative policy optimization (GRPO) with pruning. The policy learns from mea- sured pipeline costs. A byte lower bound prunes branches that cannot improve the best verified candidate. OCRL recommends a compression operator pipeline for user-specified objective weights. It seeks a lower objective value than the com- pared baselines on real-world datasets.

Primary area:
reinforcement learning
Keywords:
compression, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CN8AOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the selection of lossless compression pipelines for numerical data under different priorities for compressed size, compression time, and decompression time. It proposes a reinforcement-learning-based method that searches reversible operator trees and uses a byte lower bound to prune unpromising branches. The paper compares weighted objective values across six preference settings on a selected test subset from five datasets.

Critical strengths

The paper combines adjustable objectives for compressed size, compression time, and decompression time with reversible operator composition and certified pruning, providing a unified way to search for pipelines under different application priorities.

Critical weaknesses

The main results use a post-hoc selected test subset, and the relationship between the interim selector used in the experiments and the full reinforcement-learning method is unclear. This makes it difficult to assess the proposed method’s performance on the complete test set.

Questions for authors
  1. Could the authors clarify which implementation produced the OCRL results in Tables 1 and 2? Section 5.1 describes a deterministic weight-gated interim selector, while Section 5.3 describes a frozen weight-by-type selector. Are these results produced by the full method described in Section 4, including GRPO training and pruning? If not, please report the results of the full method separately.
  2. Could the authors provide a fuller assessment of generalization? Table 1 reports 64 test columns selected post hoc from cases favorable to the interim selector. Please report results on the complete held-out test set and for each dataset, and clarify whether the evaluated weight combinations were seen during training. Results on unseen weights and datasets would also be helpful
2 comments

Critical Weaknesses

W1. We will replace the 64-column claim in Section 5.1 with complete predefined-partition results, mark the subset diagnostic, and identify implementations beside OCRL labels and in captions.

On the 20-dataset partition, the frozen 18-entry weight-by-type selector and direct Zstd share inputs, executable, and complete containers across 448 columns and 486 tasks. Dataset-equal J is lower for the selector at all six weights by 4.70% to 34.67%, with wins on 12, 20, 19, 15, 13, and 15 datasets. Codes 100,010,001,550,505,433 denote (1,0,0),(0,1,0),(0,0,1),(.5,.5,0),(.5,0,.5),(.4,.3,.3). This comparison fixes E0=15.1598 and D0=7.0232 ns/B.

The full partition uses the weight-by-type selector. The submitted weight-100 row uses a hybrid selector. Neither is GRPO evidence.

Questions For Authors

Q1. We will report implementation, applicable checkpoint, selection procedure, candidate-opportunity budget, and online cost scope beside Tables 1-2 and in Sections 5.1-5.3. We will define the revised gate in Section 4.2 and add its ablation in Section 5.3.

Submitted entryMeasured implementation
Table 1 weight 100frozen hybrid selector
Table 1 other five weightsfrozen 18-entry weight-by-type selector
Table 2 both columnsweight-by-type routes, conditioned versus fixed 433 route

The separate factorial uses round-30 seed 0, selected on 19 validation inputs from three seeds without test outcomes. Learned and heuristic orderings use a cap of 16 non-refilling opportunities on 29 frozen slices from 20 test datasets, each with five timing repeats. Some heuristic frontiers end at nine. E includes per-request model loading, search, and final encoding. Native selection includes model loading and search, excluding final encoding and IPC.

Gate ON reduces E by 15.25% with learned ordering and 21.25% with heuristic ordering. Native selection time falls at every weight by 13.44% to 21.53% and 17.59% to 30.51%. Learned J remains higher at all six weights, by 0.14% to 21.31% with the gate OFF and 0.27% to 21.31% with it ON. Under shared validation-frozen task-bucket caps, the gate lowers E by 5.94% and 9.75% while using 26.55% and 14.11% more opportunities. These caps permit overshoot and do not match actual runtime. With the gate ON, learned beats heuristic only at 010, by 7.89% in E and 0.0267% in J, and has higher J at the other weights.

E/slot effects use five-run task medians, dataset task means, then equal dataset and weight means. The gate saves evaluation under either ordering without establishing learned superiority. The factorial disables the submitted certified subtree rule. The revised gate bound covers one same-sample evaluation, not deployment J or a subtree. B_raw counts pre-transform numeric bytes. E and D use ns/B_raw. D is internal cold decode excluding startup and I/O. This panel fixes E0=6.3754 and D0=2.4286 ns/B. Its J is not comparable with the selector study.

Q2 .We will add the complete selector table and training scope to Section 5.1 and a complete-results appendix.

Cells give selector/Zstd J per dataset. Values below 1 favor the selector.

Dataset100010001550505433
Air-Quality.7095.4730.6886.8060.7672.7700
Arade.7739.6725.8783.97411.0400.9835
Bar-Crawl.8100.75271.0117.94961.0033.9709
Bird-migration1.1089.7112.92951.0038.9706.9852
Comments1.1871.6140.7922.8291.9239.8606
IGlocations2.9744.6714.7505.9946.9372.9150
Metro-Traffic1.0441.6858.7496.8338.8637.8430
NYC Yellow Taxi Trip2.5935.7554.92271.11061.12561.0311
RealEstate1.8026.7092.8751.98811.0635.9878
RealEstate21.0355.6599.8593.99261.06751.0059
Romance.9440.6035.84931.0335.99381.0186
Telco.8065.7439.8702.9804.9589.9673
TrainsUK2.5785.7445.9470.9304.9792.9797
Uberlandia1.6753.7265.91271.02331.04311.0150
Wins.8336.7226.72421.0723.91571.0127
YaleLanguages1.0741.6807.8403.8989.9098.9260
medium posts.8665.3176.9119.6863.9291.7772
meteoblue.6499.6173.7839.98601.0091.9778
poisson timestamps.8925.6177.7722.6930.7447.7026
rating1.3112.6590.8110.8935.9177.8934

The overall reduction is 1−mean_d(Js,d)/mean_d(Jz,d), not the mean of displayed ratios. At weight 100, the reduction is 5.94%, the ratio of means is .9406, and the mean row ratio is 1.0336.

Training, validation, and test have pairwise-disjoint source groups, dataset identities, and payload hashes. All six weights were seen in training. The 29 historical slices were held out from this model's training and validation. They provide seen-weight evidence for one policy, without unseen-weight or complete-partition GRPO evidence. The fixed selector provides complete-partition coverage.

I thank the authors for their detailed responses and additional experiments, which have addressed my questions.

Reviewer_fMBrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

this paper model lossless numerical compression as an operator tree that satisfies specific type and reversibility constraints. The optimization objective is defined as a weighted sum of the compressed size, normalized encoding time, and decoding time.

Critical strengths
  1. the definitions in the text are highly rigorous.
  2. the article is well-structured, with clear logic.
Critical weaknesses
  1. the core contributions regarding learning strategies and search strategies lack direct empirical validation. Section 4 introduces a trained conditional GRPO, an eight-element trajectory grouping method, and a safety-based pruning strategy. However, sec 5.1 employs a deterministic, weight-gated intermediate selector, while sec 5.3 utilizes a fixed, type-based weighted selector.

  2. the methods for filtering results and the criteria for calculating costs are inconsistent. section 5.1 explicitly states that the configuration using 64 columns was selected post-hoc, a setup that favors the intermediate selector given the weights (0.4, 0.3, 0.3). Furthermore, the "Best single" result is based on an entire column of data, rather than being compared across identical test subsets.

Questions for authors
  1. in Table 1, please clearly indicate, which data points were obtained using the complete GRPO system with pruning, and which originated from the deterministic intermediate selector.

  2. given consistent constraints on the number of candidate evaluations and the total runtime budget, what is the performance improvement of this complete framework compared to (a) GRPO without pruning, and (b) a pruned baseline that lacks the learning module?

1 comment

Critical Weaknesses

W1

We will separate selector, learning, and gate evidence in Sections 5.1-5.3 and captions.

Submitted tables use fixed selectors. On 29 frozen slices from 20 datasets, four arms cross learned/heuristic ordering and gate OFF/ON. The gate saves time for both, while learned J is higher at all six weights under both gate settings at the fixed opportunity cap.

Five runs time one validation-selected checkpoint. The submitted subtree rule is disabled.

The formulation defines typed legal pipelines, complete frames, and bitwise-recovery acceptance. Its subtree bound assumes certified byte retention in the declared search space. These conditions do not establish GRPO superiority.

W2

We will replace the post-hoc 64-column claim with complete selector results in Sections 5.1-5.2.

The predefined partition has 20 datasets, 448 columns, and 486 weight-by-type-versus-Zstd tasks. Both share slices, executable, full frames, CPU, one warmup, and five repeats. Dataset-equal J falls by 4.70% to 34.67% across six weights. We will remove the unmatched Best single ranking and separate offline costs from online E using Q2's common-volume amortization.

Questions For Authors

Q1

We will annotate OCRL labels and captions in Sections 5.2-5.3 with implementation, applicable checkpoint, selection procedure, opportunity cap, and online costs.

Submitted entryMeasured implementation
Table 1 at (1,0,0)Frozen hybrid selector
Table 1 at other five weightsFrozen 18-entry weight-by-type selector
Table 2 weight-conditionedSame 18-entry selector
Table 2 fixedMeasured (0.4,0.3,0.3) route re-scored

None is full GRPO with pruning. This ablation uses round-30 seed 0, selected on 19 validation inputs without test outcomes. All full-partition rows use weight-by-type selection.

Q2

We will specify shared settings and cost accounting in Section 5.1.

Selector versus Zstd uses E0/D0=15.1598/7.0232 ns/B. The 29-slice ordering study uses 6.3754/2.4286. J is not compared across calibrations. Codes 100/010/001/550/505/433 give (a,b,c) after division by digit sum. Time terms use T/(T+T0).

B_raw is pre-transform numeric bytes. Rho is complete-frame bytes/B_raw. E includes selection once, encoding, and packaging in ns/B_raw. Four-arm selection includes per-request model loading and search. D is internal cold-decode ns/B_raw, excluding startup and I/O. Offline work is separate. Amortization uses E_online+T_offline/B_deploy, with T_offline in ns and B_deploy a common cumulative deployment volume in original numeric bytes. The complete-partition all-method GRPO comparison is unavailable.

Q3

We will report four-arm results in Section 5.3 with explicit budgets. Equal-opportunity results are complete. Runtime-matched and random-matched comparisons remain open.

L0/L1 use one learned checkpoint with gate OFF/ON. H0/H1 use heuristic ordering. Arms share inputs, catalog, beam, initial candidate, and verifier. S=16 caps non-refilling opportunities, not completed evaluations. Some heuristic frontiers end at nine. Cells report lower-is-better J using five-run task medians, within-dataset means, then equal dataset means.

WeightL0L1H0H1
100.760462.760462.626872.626872
010.994243.992440.992856.989808
001.851224.851429.844925.843789
550.878844.878001.815494.812507
505.828449.828522.752015.751346
433.873273.873035.826498.824949

Gate ON cuts native selection time by 13.44% to 21.53% for learned and 17.59% to 30.51% for heuristic ordering. Selection includes model loading and search, excluding final encoding and IPC. Across 870 searches per arm, completed evaluations fall from 13,770 to 2,684 and 13,385 to 2,452. Expanded-state counts are unchanged. Failures use slots but are excluded from these counts. Across six weights, E falls by 15.25% and 21.25%. An exploratory post-hoc dataset-cluster bootstrap with 20,000 resamples gives 95% intervals [14.14,16.38]% and [17.24,26.25]%.

OFF does no gate work. ON retains ties and descendants and bounds only the same-sample candidate score, not deployment J or subtrees. Learned ON worsens J by 0.0241% at 001 and 0.0088% at 505. Heuristic ON improves five time-weighted objectives and leaves pure size unchanged.

The second condition shares validation-frozen task-bucket caps. Boundary overshoot prevents equal runtime. Learned ON lowers J versus OFF at 100/550/505/433 but worsens cold D by 1.25% to 2.89%. Pure-size gain affects 2/20 datasets. Only soft-cap 010 favors learned over heuristic, with ON lowering E by 7.89% and J by 0.0267%.

The earlier subtree ON/OFF test changes paired-relative J by at most 0.093% and native time by -0.026% to +0.123%. Both earlier random seeds have lower descriptive J than learned seed 0 at four positive-size weights. Random was not rerun under the new budgets.

Reviewer_csviOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OCRL for composing lossless compression pipelines under user-specified trade-offs between compressed size, compression time, and decompression time. It represents pipelines as reversible operator trees and combines a preference-conditioned GRPO policy with a certified byte-based lower bound for search pruning. The paper reports comparisons under six objective weight settings on a selected subset of numerical columns.

Critical strengths
  1. The adjustable objective captures practical trade-offs between compressed size, compression time, and decompression time, allowing the selected pipeline to reflect application-specific requirements.

  2. The formulation carefully handles lossless reconstruction through type constraints, explicit reconstruction metadata, complete-frame size accounting, and bitwise round-trip verification. These requirements give operator composition a clear notion of correctness.

Critical weaknesses
  1. The evaluation is based on a post-hoc subset selected from cases favorable to the interim selector, as explicitly stated in Section 5.1. This introduces selection bias and prevents the reported results from establishing general performance. Evaluation on the full, predefined test set is needed.

  2. The connection between the proposed GRPO method and the evaluated implementation is unclear. Section 5.1 describes a deterministic weight-gated interim selector, while Section 5.3 refers to a frozen weight-by-type selector. The paper needs to explain how these selectors relate to the proposed GRPO policy and provide direct evidence for the learned method's effectiveness.

  3. The practical benefit of pruning is not demonstrated. There are no measurements of pruning rates, search-node counts, or search time, nor a comparison with pruning disabled. The bound also cannot prune for time-only objectives, so its effectiveness across different objective weights needs to be evaluated.

Questions for authors

Could the authors compare all methods on identical test inputs and clarify how selection and offline fitting costs are accounted for and amortized? In Table 1, the single-algorithm reference has better raw size and timing metrics under the balanced weights but a worse J. Does OCRL retain its advantage under a common cost-accounting protocol?

1 comment

Critical Weaknesses

W1. We will replace the favorable-subset performance claim in Sections 5.1 and 5.2 with complete-partition selector results.

The predefined partition contains 20 datasets, 448 columns, and 486 tasks, all included without outcome filtering. The frozen weight-by-type selector and Zstd share inputs, complete containers, CPU, and five measurements. Dataset-equal J reductions are 5.94%, 34.67%, 15.74%, 8.20%, 4.70%, and 7.97% in the weight order in W3. They are computed as 1-mean_d(J_selector)/mean_d(J_Zstd).

The 64-column analysis remains diagnostic. Learned search is evaluated separately on 29 frozen slices from these 20 datasets.

W2. We will identify the actual implementation beside each OCRL label and in table captions and Sections 5.1-5.3.

Table 1 uses a hybrid selector for pure size and an 18-entry weight-by-type selector otherwise. Table 2 re-scores weight-specific and fixed (0.4,0.3,0.3) routes from the latter. None of these entries uses trained GRPO. The complete-partition comparison in W1 uses the weight-by-type selector at all six weights.

Four arms compare GRPO-learned and heuristic ordering with candidate gate OFF/ON. Validation selected round-30 seed 0. At a cap of 16 non-refilling candidate opportunities, learned ordering has higher J at all six weights under either gate setting, by 0.27% to 21.31% with the gate ON. We will limit empirical claims to selector gains and gate savings.

W3. We will add the revised candidate gate in Section 4.2 and its four-arm ablation in Section 5.3, separately from the submitted subtree rule.

With matched ON/OFF candidate-slot traces, the gate reduces native selection time at every weight for both orderings. Native selection includes model loading and search, excluding final encoding and IPC. Over 870 measured searches per arm, learned ON skips or aborts 11,086/13,920 opportunities and heuristic ON 10,933/13,815. Completed evaluations fall from 13,770 to 2,684 and from 13,385 to 2,452, respectively. Expanded-state counts remain 13,770 and 13,385. Failures consume slots but are excluded from completed evaluations and expanded states.

Weight (a,b,c)Learned time savedHeuristic time savedLearned J reduction
(1,0,0)19.74%30.04%0.0000%
(0,1,0)21.53%30.51%0.1814%
(0,0,1)13.44%21.07%-0.0241%
(.5,.5,0)21.03%25.29%0.0959%
(.5,0,.5)14.93%17.59%-0.0088%
(.4,.3,.3)16.57%22.02%0.0273%

Aggregation uses five-run task medians, task means within datasets, then an equal mean over 20 datasets. Savings are (OFF-ON)/OFF after aggregation. Heuristic J reductions range from 0 to 0.3663% with no weight-average regression.

The gate uses L=arho+bE_cand/(E_cand+E0)+c*d_cand/(d_cand+D0). E_cand is that candidate's encode-time sample. d_cand starts at zero and increases with elapsed time during that same decode. Both use ns/B_raw. Strict L>incumbent score ends only this candidate's evaluation, preserving ties and descendants. All old structural bounds are disabled, and OFF performs no gate work.

The submitted storage-derived bound is zero when a=0. The revised gate can stop time-only candidate evaluations. Its bound covers one sample, not deployment J, timing medians, or a subtree. Two J reductions are negative. Under a separate shared soft-time cap, ON explores more candidates. Learned ON beats heuristic ON only at (0,1,0), lowering E by 7.89% and J by 0.0267%. Safe-boundary overshoot prevents an equal-actual-runtime claim.

Questions For Authors

Q1. We will report raw rho, E, D beside J and label Best single as a non-comparable whole-column reference in Sections 5.1-5.2.

Under common online accounting, the complete-partition selector retains its advantage over Zstd. We will remove rankings across unmatched inputs. In Table 1, Best single uses whole columns and a sweep charge of 21481.812 ns per original numeric byte. The selector uses test slices and 17.017 ns per training byte for fitting. Removing these charges changes their respective J values from 0.4277/0.3959 to 0.1443/0.3111. These unmatched inputs still prevent a valid ranking.

Rho is complete compressed bytes/B_raw, with B_raw measured before transforms. E includes selection once, final encoding, and packaging in ns/B_raw. GRPO E also includes per-request model loading. Its D is decoder-internal cold time in ns/B_raw, excluding startup and I/O. Offline work will be listed separately. Amortization requires E_amortized=E_online+T_offline/B_deploy, with T_offline in ns and B_deploy a common cumulative volume of deployed original numeric bytes.

E0/D0 are shared within each comparison, at 15.1598/7.0232 ns/B for selectors and 6.3754/2.4286 for GRPO. Their J values are not directly comparable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.