acceptodds
Under review as a conference paper at ICLR 2027

Who Needs More Bits? Task-Aware Quantization for Multi-Source Collaborative Inference

Abstract

Collaborative AI models increasingly integrate multiple views, modalities, or agents to solve a shared task, yet different information sources can contribute unequally to the final prediction. This raises a fundamental question for model quantization: how much precision should each source receive, and what determines its precision requirement? We study this problem from an information-theoretic perspective. We first establish a local-to-global perturbation characterization that connects source-wise parameter quantization errors to collaborative output distortion. Building on this result, we derive lower and upper rate-distortion bounds for task-aware multi-source model quantization by jointly accounting for source-specific parameter statistics and task contributions. We further characterize heterogeneous source contributions from a partial information decomposition perspective. Based on these results, we derive an explicit precision-allocation law and develop a lightweight precision-allocation rule that maps the theoretical solution to hardware-supported bit-widths. Experiments across multiple collaborative models, quantization methods, and practical resource constraints show that the derived bounds capture practical performance trends and effectively guide source-wise precision allocation.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
collaborative inference, task-aware quantization, edge intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jr8ROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies task-aware precision allocation for multi-source collaborative inference. It derives parameter-to-output perturbation bounds and rate-distortion bounds under an exponential parameter-magnitude model, then combines Shapley-based source contributions with parameter statistics to guide bit-width allocation. Experiments on image-region classification tasks report improvements over equal-bit allocation and performance close to exhaustive search, with additional evaluation under latency and energy constraints.

Critical strengths
  1. The paper studies how to allocate limited precision across sources that contribute unequally to the task, which is a relevant problem for compression in collaborative inference.

  2. The allocation rule combines source contributions with parameter statistics in a simple and interpretable form. Once these quantities are estimated, the allocation itself is cheap to compute.

  3. In the reported experiments, the method outperforms equal-bit allocation and comes close to exhaustive search.

Critical weaknesses
  1. The theory does not connect source contributions to quantization sensitivity. Proposition 1 uses coefficients ChLkAk(l)C_hL_kA_k^{(l)}, whereas Proposition 2 defines Dα\mathcal{D}_\alpha (Eq. 21) using Shapley weights αk\alpha_k, without establishing a relationship between them. The surrogate also aggregates parameters across layers without accounting for their different sensitivities. Moreover, Eq. 18 relies on a first-order approximation without controlling the remainder. Although parameter distortion is presented as a surrogate, the claims about what determines precision requirements extend beyond the analysis.

  2. The allocation rule optimizes only the lower bound. Minimizing DLD^L does not generally minimize actual distortion or preserve configuration rankings. Thus, the uniform-precision optimality condition applies to the lower-bound surrogate, not necessarily to task performance. At high rates, DU≈2DLD^U\approx2D^L could support an approximation argument, but its applicability at low bit-widths is not established. Algorithm 1 also leaves the distortion objective unspecified, while Appendix F uses DL+DUD^L+D^U.

  3. The connection between the rate-distortion bounds and practical quantizers is insufficiently justified. The upper bound concerns the Shannon rate-distortion function and need not bound uniform or K-means quantization error. Since the theoretical source is ∣W∣|W|, the mapping from magnitude-coding rate to the bit-width of signed weights also needs clarification. Figure 5 labels the bounds as “Fitted”; if they are fitted to measured distortion rather than computed from independently estimated λk\lambda_k, their agreement provides limited validation.

  4. The baselines do not isolate the source of improvement. Section 6.2 states that the branches have similar parameter statistics, so allocation may be driven primarily by αk\alpha_k. Figure 1 also assigns zero bits to one source, effectively removing it. Comparisons with Shapley-only allocation, direct quantization-sensitivity allocation, and source pruning followed by equal-bit allocation are needed to distinguish the benefits of the proposed framework from simpler alternatives.

  5. The efficiency and deployment claims need further evidence. Table 1 does not clarify whether allocation time includes Shapley estimation, which requires coalition evaluations. The “no repeated inference” claim should therefore be qualified. Appendix F assumes latency and energy scale linearly with bit-width, but it is unclear whether Table 2 uses modeled or measured constraints. Supported kernels, hardware measurements, and the treatment of communication costs should be specified.

Minor issues

  • Figure 1 reports a Shapley value of 0.35 for source 3, whereas Figure 3 and the text report 0.45.
  • Figure 2(d) is labeled GPT-3, but the cited model is GPT-Neo 125M.
  • Appendix E (line number 859) contains a broken reference (“Figures 11 and ??”).
  • Appendix C claims that Shapley contributions “naturally capture” unique, redundant, and synergistic information, but no formal connection to these PID components is established.
Questions for authors

My main concerns are the missing link between the sensitivity analysis and the Shapley-weighted surrogate, and the lack of ablations showing that the framework adds value beyond Shapley-based allocation or pruning. I would consider raising my score if the authors can address the weaknesses above and answer the two questions below convincingly.

  1. In Figure 5, are the curves labeled "Fitted DLD^L" and "Fitted DUD^U" computed directly from Eq. 4 using λk\lambda_k estimated from the weights, or are they fitted to the measured distortion? Please also report the estimated λk\lambda_k for each source and explain how they were obtained.

  2. Since Section 6.2 states that all sources have similar parameter statistics, how does the proposed method compare with (i) allocating bits based on αk\alpha_k alone, and (ii) removing sources with near-zero contribution and then using equal bit-widths for the rest, under the same budgets as in Table 1?

Reviewer_cnvmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a task-aware quantization framework for multi-source collaborative inference, combining rate-distortion theory with Shapley-based source contributions to guide precision allocation. The authors derive theoretical distortion bounds and a closed-form allocation rule, followed by a dynamic programming algorithm for discrete bit-width selection. Experiments demonstrate improved accuracy and allocation efficiency under bit-budget, latency, and energy constraints.

Critical strengths
  1. The paper provides an interpretable framework connecting source contributions, parameter statistics, and quantization precision. The derived allocation rule offers theoretical guidance for multi-source mixed-precision quantization.

  2. The proposed method consistently outperforms equal-bit allocation, particularly under tight bit budgets. It also achieves performance close to exhaustive search with substantially lower allocation overhead.

Critical weaknesses
  1. The paper uses Shapley-based task contributions to weight quantization distortion without rigorously establishing their relationship to quantization sensitivity. This weakens the theoretical connection between the local-to-global distortion analysis and the proposed precision-allocation objective.

  2. The experiments primarily construct multiple sources by partitioning a single image, which may not adequately represent realistic heterogeneous collaborative inference. Moreover, comparisons with established sensitivity-aware mixed-precision allocation methods are missing, making it difficult to assess the method's advantages over stronger baselines.

Questions for authors
  1. What is the theoretical relationship between Shapley-based task contributions and source-specific quantization sensitivity? Under what assumptions is using Shapley contributions as distortion weights justified?

  2. How does the proposed method compare with source-wise adaptations of Hessian-based or gradient-based mixed-precision allocation? Such comparisons would help establish whether task-aware contributions offer advantages beyond conventional sensitivity measures.

  3. How does the method generalize to heterogeneous sources with different architectures or modalities? Additionally, how is the 0-bit allocation shown in Figure 1 implemented in practice?

Reviewer_kshvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers how to allocate weight precision across source-specific branches in collaborative inference. It combines a parameter-to-output perturbation analysis, an exponential model of parameter magnitudes, and Shapley estimates of source contributions. A weighted rate–distortion surrogate leads to a continuous allocation rule, followed by discrete optimization over supported bit-widths. The experiments report sizeable improvements over equal-bit allocation and performance close to exhaustive search, together with additional distortion analyses and resource-constrained evaluations.

Critical strengths
  1. The paper formulates a relevant deployment problem and develops a manageable allocation procedure. Treating source branches separately is sensible when their contributions to the final prediction differ. The resulting continuous rule is interpretable, and the discrete refinement accommodates a restricted set of available precisions.

  2. The reported allocation results are encouraging, including the comparison with exhaustive search. At a four-bit average budget with uniform quantization, Table 1 reports 75.8% accuracy, compared with 66.8% for equal-bit allocation and 76.4% for exhaustive search. The six-bit result is similarly close to the exhaustive reference. This provides meaningful evidence that the proposed rule finds good configurations in the evaluated setting.

Critical weaknesses
  1. The distinctive value of source-presence attribution is not yet established against relevant allocation alternatives. Shapley-guided precision selection already appears in SMPQ and CoopQ. These methods use different games: bit-width operations or high/low-precision states, whereas this paper measures the utility of including a source. That is a substantive distinction, but source usefulness and sensitivity to quantization need not induce the same ordering. Table 1 compares against equal allocation, random allocation, rounding of the proposed solution, and exhaustive search; it does not show whether the chosen source-utility signal improves upon a source-level quantization-sensitivity score. Given the existing work, this is important for assessing what the proposed formulation adds. A focused comparison in the current four-source setting would be more informative than expanding to many additional benchmarks.

  2. The evidence supporting the predictive and efficiency claims is not sufficiently transparent. Figure 5 labels the bounds as “Fitted,” without explaining whether their parameters come from the original weights or from the displayed quantization errors. The distinction matters: fitting observed errors can demonstrate descriptive agreement, but does not establish independent prediction from parameter statistics. Likewise, Table 1 does not make clear whether the 1.02 ms allocation time includes Shapley estimation and other calibration. These issues do not negate the reported accuracy gains, but they affect two advantages emphasized by the paper: theoretical guidance and inexpensive allocation.

Questions for authors
  1. What is the practical advantage of source-presence Shapley utility over a score based directly on quantization-induced output or loss changes? Please clarify the distinction from the games used in SMPQ and CoopQ. A source-level sensitivity comparison on the existing setup, or existing evidence addressing this choice, would help resolve my main concern.

  2. How are the exponential-distribution parameters, normalizations, and any fitted constants in Figure 5 obtained? Were the curves predicted from the original weights or fitted to measured distortion? Please also explain why the displayed bounds differ between the two quantizers applied to the same source model.

  3. Which operations are included in the reported allocation time? Please separate attribution/calibration costs from the final optimization, and clarify which latency and energy quantities in Table 2 were measured and which were calculated from the models in Appendix F.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.