acceptodds
Under review as a conference paper at ICLR 2027

On What We Can Learn from Low-Resolution Data

Abstract

Artificial intelligence systems typically rely on large, centrally collected datasets, a premise that does not hold in many real-world domains such as healthcare and public institutions. In these settings, data sharing is often constrained by storage, privacy, or resource limitations. For example, small wearable devices may lack the bandwidth or energy capacity needed to store and transmit high-resolution data, leading to aggregation during data collection and thus a loss of information. As a result, datasets collected from different sources may consist of a mixture of high- and low-resolution samples. Despite the prevalence of this setting, it remains unclear how informative low-resolution data is when models are ultimately evaluated on high-resolution inputs. We provide a theoretical analysis based on the Kullback–Leibler divergence that characterises how the influence of a datapoint changes with resolution, and derive bounds that relate the relative contribution of high- and low-resolution observations to the information lost under downsampling. We validate these results in simulation and further show empirically, using both a vision transformer and a convolutional neural network, that low-resolution data can improve performance when high-resolution data is scarce, across controlled benchmarks and a naturally heterogeneous medical dataset.

Primary area:
learning theory
Keywords:
KL-divergence

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mo2VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the contribution of low-resolution training data to models evaluated on high-resolution inputs. The authors consider a mixed-resolution learning setting, motivated by practical constraints such as storage limitations. The results suggest that using more low-resolution data can improve high-resolution classification performance, particularly when high-resolution training samples are scarce.

Critical strengths
  1. The paper addresses an important problem in machine learning, that data may be collected at heterogeneous resolutions. The assumption that high- and low-resolution observations are unpaired is realistic.
  2. The authors provide quite comprehensive theoretical analysis based on KL divergence,this offers an interesting perspective on understanding the contribution of observations at different resolutions.
  3. The experimental contains image and audio classification, including a multi-institutional medical dataset. The comparison provides useful observations regarding distribution shifts.
  4. The supplementary investigates more impacts on other parameters.
Critical weaknesses
  1. I think there is a discrepancy between the paper's theoretical and empirical evaluation. The theory measures parameter shifts between resolutions of the same observation, but the experiments measure downstream accuracy after adding distinct low-resolution samples. I found their link is still unclear.
  2. The proposed mixed-resolution strategy resembles data augmentation in training (e.g., multi-scale, corruption-based data augmentation). The experiments didn't show the benefits of this strategy. As the paper lacks direct baseline comparisons to standard downsampling or resolution-degradation applied to existing high-resolution samples, it remains unclear whether the performance gains are driven by increased dataset size.
  3. The experimental evaluation relies on simple, low-resolution classification benchmarks (like 32×32 CIFAR). To convincingly demonstrate general applicability, the paper requires experiments on genuinely high-resolution, complex datasets, for example such as 3D medical imaging, video, where resolution degradation physically destroys task-relevant fine structures.
Questions for authors

The authors can see the weaknesses for questions.

2 comments

We thank the reviewer for appreciating the realistic problem setting and the comprehensive theoretical analysis.

  • W1: You are completely right, we have addressed this in W1 Reviewer Bb5J W1

  • W2: You are right that this is not mentioned in the paper. To make it more clear, we have added a sentence in the Empirical Setup section that mentions our data augmentation explicitly; this was lacking in our original submission. To summarise our data augmentation, the Subset experiments include several data augmentation approaches to ensure we obtain good performance, and such that the improvements in the Ratio experiments can be isolated to the addition of the low-resolution samples. Appendix D.2 details that we add randomcrop, random erasing, and color jitter that all degrade the data in ways similar to downsampling. We further also train with CutMix and Mixup.

  • W3: You are right that many of our results are based on experiments on CIFAR10 and CIFAR100. We emphasise that we also report results based on the real-world medical fed-isic2019 dataset that is 224×224224 \times 224 in dimensions and is therefore genuinely high-resolution. Overall, we believe the empirical evidence is sufficient to demonstrate that incorporating low-resolution data during training can provide meaningful benefits when high-resolution data are scarce.

We believe that we have addressed all of your concerns, and if so hope you would consider updating your recommendation accordingly.

Dear Reviewer mo2V. We thank you again for your review. As the discussion period is coming to an end, we would be grateful for your feedback on whether our response and the revised PDF have addressed your concerns, or if there are any points that would benefit from additional clarification.

Reviewer_Bb5JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies mixed-resolution supervised learning, where training data contain both high- and low-resolution observations, while evaluation is performed exclusively on high-resolution inputs. Under a Gibbs assumption on the parameter distribution, the authors quantify sample influence through the KL divergence induced by adding an observation. They derive an exact expression, a variance-based approximation, and approximate bounds involving the residual between high- and low-resolution representations. Experiments on CIFAR, AudioMNIST, and heterogeneous medical data show that additional low-resolution observations can improve high-resolution classification, particularly when high-resolution training data are scarce.

Critical strengths
  1. The problem setting is interesting. Exploiting unpaired low-resolution observations under storage and transmission constraints is a useful question.

  2. The analysis offers an interpretable perspective: the effect of resolution depends on both the discarded information and the model’s sensitivity to that information. The exact Gibbs KL identity and the linear-regression example help explain the proposed influence measure.

  3. The evaluation includes both CNNs and ViTs, controlled resolution changes, and a heterogeneous medical dataset. Reporting the nonmonotonic behavior of the ViT under increasing external donations is informative and appropriately qualifies the benefits of additional data.

Critical weaknesses
  1. My main concern is that the theory and experiments study different notions of usefulness. The theory measures how much a sample changes the parameter distribution, whereas the experiments measure classification accuracy. A sample can substantially change the parameters without improving generalization. The paper does not show whether its influence measure predicts which low-resolution samples are actually helpful.

  2. It is difficult to tell when the approximate bounds are reliable. Proposition 4 drops higher-order cumulants, and Propositions 5–6 also discard the input Taylor remainder. Neither error is quantitatively controlled. This matters for settings such as downsampling from 32×32 to 8×8, where the change is substantial. The simulations are encouraging, but do not establish how broadly the approximations hold.

  3. The Gibbs assumption is not sufficiently connected to the training procedure. Randomness from initialization and minibatch sampling does not imply that AdamW produces Gibbs-distributed parameters. Similarly, evaluating the Gibbs-derived expression over independently trained networks does not establish that it equals the KL between the actual parameter distributions before and after adding a sample. This limits what the simulations validate.

  4. The practical comparisons could be stronger. Adding distinct labeled low-resolution samples to a small high-resolution dataset is useful, but the more interesting question is whether this is preferable to spending the same storage budget on fewer high-resolution samples or more intermediate-resolution samples. Figure 4 reports storage costs, but does not directly resolve this comparison. Training compute and the number of updates should also be accounted for.

  5. The medical experiment is harder to interpret because donor samples are added starting with minority classes. Some of the improvement may come from better class coverage. The full-resolution donor control helps, but a randomized donation order would make the conclusions more convincing.

Questions for authors
  1. Does the proposed influence score correlate with the actual change in high-resolution test performance after adding samples? Even a small controlled experiment would help connect the theory to the main empirical claim.

  2. How large are the cumulant and Taylor approximation errors at the resolutions used in the experiments? Can the authors identify settings where the approximations break down?

  3. In the repeated-training simulations, what evidence supports treating the trained parameters as Gibbs samples? Are the reported “true KL” values directly estimated between parameter distributions, or obtained solely by evaluating Proposition 3?

  4. Does mixed-resolution training still outperform the alternatives when storage and training compute are matched? For the medical experiment, do the gains persist when the donation order is randomized?

4 comments

We thank the reviewer for highlighting the interesting problem setting and the interpretable perspective offered by the theoretical analysis.

  • W1: This is a very good point, and you identify a missing link in our paper. To investigate we setup a new experiment where we measure the correlation between the KL-influence and the generalization error for 120 example points. We find a Spearman correlation coeffiecient of 0.53 between the KL and the accuracy so our influence measure does indeed identify helpful points. We have added section 7 in the new pdf that bridges the gap between theory and the empirical results. Here we show that the KL-influence measure of a datapoint can be used to indicate whether that given point will improve or worsen the model performance.

  • W2: You are correct that the approximations discard higher order terms. Appendix B provides a second order Taylor approximation of the bounds and shows through simulations that the bounds do not become tighter with this refinement (See W3). Appendix C.3 also provides a quantitative analysis on the variance approximation of Proposition 4 and shows for the worst case that the relative error stays below 11%.

  • W3: This is indeed a valid concern and thank you for the remark. After careful consideration, we agree that AdamW training with random initialisation is not guaranteed to produce samples from the Gibbs distribution. We have updated our approach so we now draw samples directly from the Gibbs distribution using the Metropolis-adjusted Langevin Algorithm (MALA) [Roberts and Tweedie, 1996]. MALA is a Langevin-dynamics sampler [Welling and Teh, 2011] whose target is the Gibbs disitribution. We can thus evaluate Proposition 3 with these Monte Carlo samples. We have updated Figure 2 (and Figure 7 in Appendix B) accordingly. After these modifications our original findings remains the same.

  • W4: This is a very good point that has not been adequately scoped in the paper. We have added an Appendix G that compares the number of floating point operations for the Ratio- and Subset experiment and how that relates to improvement in accuracy. Our new results shows that for low ratios of high-resolution data, adding large amounts low-resolution data can sometimes outperform adding a few high-resolution samples in terms of accuracy while still resulting in fewer floating point operations. Please note that objective of the mixed-resolution setup is that it is not feasible to obtain high-resolution samples directly, and the situation where a requester can ask for a specific resolution (i.e more high res data) is not considered.

  • W5: That is a good observation. To investigate, we have run the experiment again where the donors randomly donate samples. Results are added to Appendix E and shows no qualitative difference between the two donation orders. The results for very small amounts of low-resolution donor data are not ready yet, but will be included once they are.

  • Q1: See W1

  • Q2: See answer to W2. Furthermore, Appendix B.4.2 investigates the assumption imposed in Corollary 7 using the results derived from linear regression. Here we find that if the residual component corrects the prediction of the low-resolution component too much, then the assumption starts to break down.

  • Q3: See W3

  • Q4: See W4 + W5

We believe that we have addressed all of your concerns, and if so hope you would consider updating your recommendation accordingly.

Dear Reviewer Bb5J. We thank you again for your review. As the discussion period is coming to an end, we would be grateful for your feedback on whether our response and the revised PDF have addressed your concerns, or if there are any points that would benefit from additional clarification.

Thank you for running the additional experiments. The Spearman correlation of 0.53 across 120 examples gives me more confidence that KL-influence captures some information about sample usefulness, and the switch to MALA addresses my concern about how the Gibbs samples were obtained. My main remaining concern is the accuracy of the bounds under substantial downsampling. The reported error below 11% applies to the variance approximation in Proposition 4, and I’m still unsure how large the Taylor remainder in Propositions 5–6 can be when going from 32×32 to 8×8. Given that uncertainty, I’ll keep my original rating.

To analyse the size of the remainders of the Taylor approximation, we use the same simulation setup as for the simulation of the bounds. We then measure the relative error between the loss at the high-resolution datapoint ℓ(θ,xh)\ell(\boldsymbol{\theta}, \boldsymbol{x}_h) and the first order approximation, ℓ(θ,xl)+g⊤xr\ell(\boldsymbol{\theta}, \boldsymbol{x}_l) + \boldsymbol{g}^\top \boldsymbol{x}_r and the second order, ℓ(θ,xl)+g⊤xr+xr⊤Hxr\ell(\boldsymbol{\theta}, \boldsymbol{x}_l) + \boldsymbol{g}^\top \boldsymbol{x}_r + \boldsymbol{x}_r^\top H\boldsymbol{x}_r. The second order approximation is slightly better than the first order. The reason that this does not result in tighter bounds in the end, is that the second order term is introduced and subsequently also needs to be bounded using Cauchy-Schwartz. The looseness introduced by that mechanism is greater than the gain obtained by having a better local approximation of the loss. The results can be seen in the newly uploaded PDF in Appendix C.3 in Figure 13.

Reviewer_ZPMrOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the impact of low-resolution data on high-resolution test performance in mixed-resolution training scenarios. It analyzes the role of low-resolution samples from both theoretical and experimental perspectives and validates the findings on the CIFAR and medical image datasets.

Critical strengths
  1. This issue has practical significance. Exploring the value of low-resolution data in training provides useful insights for scenarios with limited data or where storage costs are a concern.
  2. Includes real-world validation. In addition to standard image classification datasets, the paper also conducts experiments on medical image datasets, which has practical value.
Critical weaknesses
  1. The theoretical validation is somewhat idealized. Some analyses rely on small CIFAR subsets, high-frequency component manipulation, PCA, and shallow neural networks. These simplified settings may not adequately capture the information loss induced by realistic downsampling or the complex relationship between high-frequency details and semantic information, limiting the empirical support for the proposed theoretical claims.
  2. Experiments show that incorporating low-resolution data does not always improve model performance; in some settings, it can even lead to a decline in performance. However, the paper fails to thoroughly analyze the relationship between these differences and model architecture, data distribution, and training configurations, and it lacks clear guidelines for predicting when low-resolution data is beneficial and when it is harmful.
  3. The experiments focused primarily on CIFAR-10, CIFAR-100, and a medical image dataset, using a limited number of CNN and ViT models. The current results are insufficient to demonstrate that the relevant conclusions can be generalized to different data distributions, model scales, and task types; the paper’s main findings have not been supported by a broader range of scenarios and experiments.
  4. The distinction between this work and existing studies on multi-resolution training, cross-resolution representation learning, and scale-consistency constraints is not sufficiently clear. The paper does not adequately establish what new insights its theoretical analysis provides beyond these related directions. Moreover, the limited benefits of the scale-consistency loss raise questions about its necessity and practical contribution.
Questions for authors

see weakness

5 comments

We thank the reviewer for highlighting the practical significance of the problem setting and the real-world validation.

  • W1: We agree that the simulations use a simple setting, and this is a deliberate choice. We have updated our theoretical validations so we are now sampling from the Gibbs posterior using the MALA algorithm (se reviewer Bb5J W3). MCMC samples are only tractable on smaller model scales and input dimensions. The DWT decomposition is used to exactly ensure xh=xr+xlx_h = x_r + x_l (Appendix C.1), and PCA is used to shrinken the input dimension and since it is a linear operator it does not disrupt the decomposition. Appendix C.3 shows that the variance approximation becomes more accurate and the bounds tighter when as model size increases, suggesting that the small-network setting is, if anything, a conservative test.
  • W2: See Reviewer Bb5J W1
  • W3: You are indeed right that only a limited number of CNNs and ViTs are tested. We want to emphasize that our study is focused on the regime of limited storage and data. It is therefore outside the scope of the paper to sweep a large range of model sizes. Moreover, the fed-isic2019 experiment does mix different data distributions as data is collected across different institutions. The empiricial evaluation serves more as an accompany to the main contribution of the paper that is the theoretical analysis.
  • W4: The first paragraph in the Related Work section outlines how this paper differs from current state of the art, and each following paragraph in the related work section (except for the one about CNNs and ViTs) ends with a sentence describing how this paper differs. Our work primary differs is that we do not assume access to paired high- and low-resolution samples, but consider the setup where a requester (e.g. for privacy reasons and limited bandwidth and battery) can only request low-resolution samples. To the best of our knowledge this is a setting that has not been previously considered nor theoretically analyzed. If you still believe we are unclear, could please elaborate and we are more than happy to answer in more detail. In regards to the scale-consistency loss, it is an ablation study that was added to incorporate prior methods. Since there was no added benefit from this additional loss, we chose to have this in the appendix.

We believe that we have addressed all of your concerns, and if so hope you would consider updating your recommendation accordingly.

Dear Reviewer ZPMr. We thank you again for your review. As the discussion period is coming to an end, we would be grateful for your feedback on whether our response and the revised PDF have addressed your concerns, or if there are any points that would benefit from additional clarification.

Reviewer_ZPMrOpenReview ↗

Response to the author

Thank you for your response. While the proposed method offers some unique insights, it still falls short of the standard required for publication at ICLR. Although you intentionally employed a simplified simulation of degradation, such scenarios hold limited practical significance in the current research landscape; extensive existing literature has demonstrated that real-world degradation is far more complex than the method presented here. Furthermore, the validation scenarios and the methodology itself are quite restricted—the work feels more like a "laboratory toy."

Please do not be discouraged; I still believe this work has merit. I look forward to seeing the authors address these issues through more systematic experimentation, which could lead to a successful publication. I am willing to raise my rating to "weak reject," but I cannot recommend acceptance for a study with such some incomplete experimental validation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.