- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes a framework for optimizing preprocessing, quantization tables, and postprocessing within the JPEG pipeline. Targeting scenarios with limited decoder-side resources, it introduces an asymmetric codec architecture for the uplink and downlink stages. Through end-to-end fine-tuning, the framework jointly optimizes rate-distortion performance and downstream task performance. The experiments demonstrate advantages in rate-distortion curves and classification accuracy after joint optimization.
Critical strengths
The learning framework is specifically designed to address the computational imbalance between edge devices and the cloud, as well as differing downstream task requirements.
Critical weaknesses
- The proposed techniques are already supported by well-established methods in prior work, including Differentiable JPEG [1] and JPNeO [2]. Related learned image compression technologies also have a standardization framework in JPEG AI [3] that provides practical guidance. These precedents limit the novelty of presenting such techniques as core contributions.
- Despite computational imbalance being a central motivation, the paper does not sufficiently explore acceleration methods on edge devices or the challenges arising from integrating them with existing optimization techniques.
[1] Differentiable JPEG: The Devil Is in the Details. WACV 2024.
[2] JPEG Processing Neural Operator for Backward-Compatible Coding. ICCV 2025.
[3] JPEG AI. https://jpeg.org/jpegai/documentation.html
Questions for authors
-
SEAOTTER-FT exhibits a substantial drop in PSNR. How does the approach address the potential loss of decoder generality caused by adaptation to a specific task? Could the authors evaluate a broader range of tasks to characterize when this adaptation improves performance and when it degrades performance?
-
What is the individual contribution of each learnable component? How does performance change when the cloud-side FRAPPE decoder is kept frozen rather than fine-tuned?
-
Does the proposed approach retain its performance advantages over a learned JPEG baseline fine-tuned for the downstream task, as well as the approaches in [1] and [2]?
[1] Differentiable JPEG: The Devil Is in the Details. WACV 2024.
[2] JPEG Processing Neural Operator for Backward-Compatible Coding. ICCV 2025.
5 comments
Thanks for the rebuttal. While some wording in my summary could be more precise, This review does not describe the submitted manuscript is not an appropriate response. Using "downlink" is not inappropriate, since the author uses "uplink" in the paper. The manuscript adopts the existing FRAPPE encoder on the sensor side and explicitly states that All SEAOTTER variants inherit the same frozen FRAPPE encoder. Therefore, I focus on the proposed learning components rather than treating the inherited sensor-side design as a new contribution. Most of the optimization is inherently on the server-side pre-processing and the quantization tables of JPEG, adapted for FRAPPE/WaLLoC. Since sensor-side encoding efficiency is inherited from FRAPPE, the major contribution of the system is therefore adapting FRAPPE/WaLLoC to the imbalanced nature of the server and app side.
The rebuttal shall focus on the improvement of the manuscript.
For Q2, the authors did not read it carefully. The only occasion that the framework finetunes FRAPPE decoder is in SEAOTTER-FT, and the ablation does not cover it (only removing the FRAPPE finetuning part in SEAOTTER-FT). This is an experiment that could strengthen the paper's technical claim of application-specific signal enhancement with specialized .
Since the authors emphasize the system contribution, the manuscript should provide stronger evidence of system implementation and overall optimization. It should assess compatibility with neural-network acceleration technology like quantization and software-acceleration technology like parallel execution, and explain how these techniques affect the claimed benefits. The current Wi-Fi, 5G, and BLE deployment suitability is based on assumed bandwidth thresholds, and the encoding throughput is measured only on a server-class CPU. End-to-End latency performance shall be measured. Both the proposed method and the baselines should use appropriately optimized implementations and inference frameworks, such as TensorRT where suitable for the target hardware, to establish a deployment-aligned comparison.
Supplementing the discussion of the variants is also helpful, such as when the server or the app has more or less computational capability, what changes to the system should be made.
Response to Reviewer 4Ynd's comment of November 11
Reviewer 4Ynd's comment of November 11 defends one word of the review's summary, restates Question 2 as a different question, and says a measurement is missing that our response of November 10 reports. It defends none of the three citations behind the review's novelty objection, and it identifies no error in any derivation, experiment, or reported number, which a soundness score of 1 requires.
The review's summary of the paper. Our response listed five inaccuracies in it. The comment contests one:
Using "downlink" is not inappropriate, since the author uses "uplink" in the paper.
We see how "downlink" follows from "uplink," and the manuscript's terminology was chosen to prevent that reading. It names three parties: sensor, cloud, and consumer. "Uplink" is the link from the sensor to the cloud. The consumer is a different device from the sensor: any machine that later reads the stored JPEG file. "Downlink" would imply that data returns to the sensor, which it never does, so the manuscript uses "consumer" throughout and does not use "downlink." The remaining inaccuracies are not matters of wording, and the comment does not address them: the summary omits the sensor encoder and the cloud decoder, places the constraint on the wrong side of the system, describes joint optimization that the zero-shot variant does not perform, and omits segmentation, zero-shot recognition, and throughput from the experiments. The comment then gives a new summary, in which the contribution is "adapting FRAPPE/WaLLoC to the imbalanced nature of the server and app side." That is the contribution the manuscript states. The ratings were assigned under the first summary.
Novelty (Weakness 1). The review states that the proposed techniques exist in Differentiable JPEG, JPNeO, and JPEG AI. Our response showed that the first trains no quantization table and declines to model file size, that the second places a network of 1,150,000 MACs per pixel at the encoder, and that the third is a separate bitstream that no JPEG decoder reads. The comment mentions none of the three, which leaves the objection without support.
Frozen decoder (Question 2).
For Q2, the authors did not read it carefully.
Question 2 reads, in full: "What is the individual contribution of each learnable component? How does performance change when the cloud-side FRAPPE decoder is kept frozen rather than fine-tuned?" Our response reported the three configurations with a frozen decoder beside the fine-tuned one, at every rate and on every task, from tables in the submitted manuscript. The comment now describes a different experiment, in which the decoder is frozen and only the JPEG components are trained on the task. Question 2 did not ask for it. The contribution of each component is nonetheless measured. The revision adds a decoder fine-tuned on the task and stored as standard ITU T.81 JPEG, with no learned JPEG component. At the 12-channel operating point and comparable file size, it reaches 47.37% SigLIP-2 zero-shot top-1 at 0.698 bpp, against 48.22% at 0.645 bpp for SEAOTTER-FT, and 27.41% ADE20K mIoU at 0.481 bpp, against 32.77% at 0.524 bpp. The decoder without fine-tuning, with standard JPEG, reaches 44.90% and 30.39% with larger files.
Edge hardware.
the encoding throughput is measured only on a server-class CPU
Our response of November 10, to which this comment replies, reports the sensor-side encoder on a Cortex-M55 microcontroller at 249 MHz: 6.18 megapixels per second (MPx/s) at the 12-channel operating point, against 0.0139 MPx/s for AVIF at the matched rate. The comment also calls the Wi-Fi, 5G, and BLE thresholds "assumed." Each is the product of a resolution, a frame rate, and a link rate from the cited measurement literature. The BLE case, 480p at 30 frames per second over 1 Mbps, requires 12 MPx/s; on the microcontroller the encoder runs at 18.4 and 11.8 MPx/s at its two lowest rates.
New requests. The comment asks for TensorRT, network quantization, parallel execution, and a discussion of variants for other compute budgets. None appears in the review, and none bears on a claim of the manuscript. TensorRT is an inference library for NVIDIA GPUs. The sensor-side encoder is a set of linear projections costing 10–100 MACs per pixel, measured on one CPU and on a microcontroller with no GPU. The baseline, AVIF, is not a neural network, so no inference framework applies to it.
Improvement of the manuscript.
The rebuttal shall focus on the improvement of the manuscript.
Our response reported three additions made in answer to this review: the microcontroller measurements, an evaluation of every fine-tuned variant on the tasks it was not tuned for, and a comparison with DeepN-JPEG, a JPEG quantization table selected for task accuracy. The comment acknowledges none of them.
Response to Reviewer 4Ynd: The review does not describe the submitted manuscript
This review does not describe the submitted manuscript. Its summary omits the system the paper proposes and attributes to it a design it does not contain. Its first weakness cites three works, none of which performs the function it is cited for. Its second weakness names no missing experiment. Two of its three questions are answered by tables in the submitted manuscript. It assigns soundness 1 and identifies no error in any derivation, experiment, or reported number. We ask the Area Chair to weigh the review according to its correspondence with the manuscript.
Paper summary. Each sentence is inaccurate. (1) The paper proposes a three-stage system: a frozen autoencoder on the sensor, whose encoder costs 10–100 MACs per pixel; a neural decoder in the cloud; and a one-time transcode of the decoded image into a standard JPEG file. The first two stages do not appear in the summary. The sensor never runs JPEG. (2) The constraint that defines the design is the sensor-side encoder, not "limited decoder-side resources." (3) There is no "downlink" stage; the word does not occur in the manuscript. (4) Rate–distortion and task performance are not "jointly optimized." SEAOTTER-ZS sees no task, label, or downstream model, and the central result is that the transcode raises accuracy in that variant (+4.03 points of ImageNet top-1 at n=12). (5) The experiments cover classification, segmentation, and SigLIP-2 zero-shot classification, with encoding and decoding throughput for every method.
Weakness 1. None of the three cited works contains a sensor-side encoder, a transcode, or a measurement of downstream accuracy for a compression system. None contains a JPEG codec whose color transform, quantization matrices, and rate proxy are learned de novo.
Weakness 2. This weakness names no acceleration method, optimization technique, or experiment whose absence affects a claim. The manuscript measures sensor-side encoding throughput for every method at every operating point: on ImageNet at n=12, 178 MPx/s on one CPU, against 5.5 MPx/s for AVIF and 25.7 MPx/s for AVIF at maximum speed. The revision adds measurements on a Cortex-M55 microcontroller (Apollo510, 249 MHz), where the encoder at n=12 runs at 6.18 MPx/s and AVIF at the matched rate at 0.0139 MPx/s (0.199 MPx/s at maximum speed).
Question 1. The drop in PSNR is the stated design of SEAOTTER-FT, which "trades pixel-domain reconstruction PSNR for higher downstream accuracy after the transcode." Generality is retained because the encoder is frozen: "the same transmitted latent decodes to different RGB outputs depending on the chosen snapshot." SEAOTTER-ZS is the general-purpose variant. The revision adds a cross-task evaluation of every fine-tuned variant: in 10 of 30 cross-task cells, all at n∈{6,9,12}, a variant tuned for another task exceeds SEAOTTER-ZS with a smaller file.
Question 2. The submitted appendix reports the frozen-decoder configurations at all five rates on all three tasks. ImageNet top-1 at n=12 and 0.109 transmit bpp: FRAPPE 56.22%, FRAPPE + ITU JPEG 60.83%, and SEAOTTER-ZS 60.25%, all with the decoder frozen; SEAOTTER-FT 69.02%.
Question 3. No JPEG codec, however its tables are obtained, operates at SEAOTTER's rates. A baseline JPEG file of these images cannot be smaller than 0.18–0.25 bpp, and SEAOTTER transmits 0.010–0.142 bpp from n=3 to n=12. At n=15, DeepN-JPEG, a quantization table selected for ImageNet accuracy, reaches 72.11% ImageNet top-1 and 33.79% ADE20K mIoU with 4:2:0 subsampling; SEAOTTER-ZS, with no task training, reaches 76.43% and 39.42%. The approach in [1] reproduces standard JPEG, which is a baseline in the manuscript. The approach in [2] cannot run on the sensor and reports no downstream result.