acceptodds
Under review as a conference paper at ICLR 2027

SAGE: Efficient Protected Vision-Language Inference on Smart Glasses

Abstract

Smart glasses are a promising platform for vision-language assistants, but private split inference remains expensive: protecting visual representations before transmission does not remove the cost of on-device vision encoding, wide protected tokens, or hardware-unaware execution. We present SAGE Smart-glasses-Aware Guarded Encoding), a hardware-aware framework for efficient protected VLM inference on smart glasses. SAGE keeps raw images and unprotected visual tokens on the trusted device, reduces visual input cost, executes shape-specialized INT8 graphs, and compresses protected representations into compact 256-dimensional INT8 packets before transmission. Unlike efficiency methods optimized only for nominal model size, SAGE selects its operating point using downstream answer quality together with measured latency, memory, and thermal behavior on real Mentra Live glasses. On a matched Qwen workload, SAGE reduces glasses inference latency by 39.18%, device-sensor temperature rise by 21.91%, and protected packet size by 96.56%, while retaining 85.62% of the protected reference's relaxed utility and comparable prompted containment. Direct glasses-to-phone tests deliver all 192 packets and reduce median packet-plus-acknowledgment time from 71.3 to 10.0 ms. The compact protected interface also transfers across four additional VLM families. These results show that private wearable VLM inference must be optimized jointly across representation protection, communication, and real-device execution rather than through model compression alone.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
smart glasses, vision-language models, privacy-preserving inference, split inference, on-device inference, efficient multimodal systems

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PfYyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to private split VLM inference on smart glasses. It keeps the prior TGAP token-protection transform, reduces the input pixel budget whild maintaining the whole struture of Qwen visual encoder. The experiment shows that the latancy reduce 39% and has less tempreature raise.

Critical strengths

Real-device measurements, e.g. latency, memory, temperature, etc. are rare and valuable.

Critical weaknesses

Privacy is assessed only by prompting the same frozen reader with containment/accuracy rates, with no trained probes or reconstruction attacks on the 256-d packet. One of the central claim of joint quality-and-hardware optimization is not validated end to end. what the reviewer understood is the utility comes from the software path, not the deployed INT8 graphs? and the Device measurements are single runs, and TGAP-HR's latency/temperature differ widely between Table 2 and Table 4, this should be explained.

Questions for authors

Are TGAP-HR and SAGE both INT8 on-device with the same runtime? What is utility of the actual deployed INT8 graph? How do conclusions change if L star is not set to 10s?

3 comments

Thank you for recognizing the real-device evaluation. We hope you can reconsider rising your rate after reading our detailed rebuttal.

Q1. Are TGAP-HR and SAGE both INT8 on-device with the same runtime?

Yes. Both run on the same Mentra Live glasses with ONNX Runtime, four CPU threads, and dynamic per-channel INT8 linear operators; geometry, normalization and unsupported operators retain native precision. Both use fixed-grid graphs, the frozen Qwen reader and the same TGAP transform. The artifact audit verifies this policy across the six-method measurement. The reported 39.18% inference gain is therefore within the same execution policy, not an INT8-versus-FP32 comparison.

Q2. What is the utility of the actual deployed INT8 graph?

We evaluated real glasses outputs on 617 paired images, holding questions and scoring fixed across all paths:

PathHR U.Rel %SAGE U.Rel %
Original software65.3756.12
Host INT8 ONNX63.2855.52
Actual glasses INT864.1855.52

SAGE's actual-device utility equals its host-INT8 value on these images; actual-minus-software is -0.60 points, paired 95% CI [-3.53,2.36]. Actual SAGE/HR P.Acc is 37.92/40.38%, P.WR 16.99/18.58%, and P.Cont 11.70/11.32%. This directly evaluates deployed representations rather than substituting software features for device outputs; the original three-seed software results retain their separate sample scope.

To address your end-to-end concern, we also completed 64 fresh file-to-answer requests: 16 images x two methods x two repeats. Each includes glasses inference, Wi-Fi ADB transfer, decoding and final generation on a resident host reader. Mean total time falls 56.26->40.71 s, a 27.64% reduction with paired 95% CI [22.34%,33.25%]. One host clock includes loading, control overhead and post-upload guard waiting. These requests connect actual deployed computation to final answers. The separate phone experiment establishes 192/192 packet deliveries; the full-response endpoint here is the host reader.

Q3. How do conclusions change if the latency reference is not 10 s?

Keeping R's formula unchanged, SAGE scores 1.409/1.409/1.369/1.141 at 5/8/10/12 s versus HR=1.000. The favorable trade-off persists across this range. At 1 s, tiny students rank higher; at 16/20 s, HR ranks higher as its latency penalty disappears. R is a requirement-based index, so we will show its sensitivity alongside raw metrics. Measured inference, temperature, packet and complete-response gains do not depend on R.

Critical Weaknesses: direct privacy attacks

We trained 36 probes: four representation conditions x mean/Transformer/metadata probes x three initializations on one deployment split. Inputs use final host-INT8 graphs after packet serialization/dequantization. Probe training/selection filters known author overlap, exact images and conservative near-duplicates against evaluation. Supported sex/income/country counts are 86/52/305.

Probe / attributeHR BA %SAGE BA %
Mean / sex68.0668.37
Mean / income85.9574.40
Mean / country43.4038.06
Transformer / sex69.1565.62
Transformer / income70.4867.27
Transformer / country39.8336.37

Observed recovery decreases in five of six content-probe comparisons. Mean country recovery falls 5.33 points, paired 95% CI [-10.16,-0.64]. This adds independent recovery evidence beyond reader prompting.

Critical Weaknesses: repeated measurements and Tables 2/4

The new same-session study covers three fixed grid pairs x four resolution/codec cells x five repeats. All 60 requests succeed under unchanged cooldown/safety rules.

GridHR latency s, mean +/- SDSAGE latency s, mean +/- SD
Small13.773 +/- 0.7695.905 +/- 0.506
Common15.145 +/- 1.9719.367 +/- 0.879
Large17.364 +/- 2.36310.668 +/- 0.352

Paired 95% latency intervals are [-8.286,-7.503], [-7.239,-3.682] and [-8.097,-4.334] s, all favoring SAGE. Mean sensor rise decreases from 8.14/8.80/8.39 C to 6.16/4.88/6.95 C. Table 2 is the six-method comparison; Table 4 is the earlier depth/precision session. We will label their distinct sessions and add these repeats: within-session comparisons reproduce the benefit without pooling absolute measurements across runs.

Ethics

New experiments reuse existing benchmark records and annotations, without new participant recruitment or wearable image capture. The design retains the trusted-device boundary. The paper's Ethics Statement addresses source identifiers, controlled artifact access and consent-aware deployment.

Reviewer_PfYyOpenReview ↗

good response but the contribution is not satisfied with ICLR quality

the author has reply to each of the review's question, the reviewer found more detailed information to support this methos is workable. but the technical contribution of this paper still not reach score 3 since the technical novlty is somehow limited.

Thank you for acknowledging that the additional experiments answer your questions and support the method's operation. We understand that your remaining concern is the level of technical contribution, rather than those validation questions.

The contribution we ask you to assess is the protected split-inference design on smart-glasses hardware: reducing visual work and post-protection transmission while retaining vision topology and the frozen reader. The measurements distinguish this choice from alternatives that appear attractive from parameter count or storage alone: depth reduction loses answer quality, while mixed INT4 is 44.38% smaller yet 64.78% slower than full-depth INT8. The resolution/codec ablation separates compute savings from communication savings, and actual-device QA connects deployed representations to downstream answering. These findings support a hardware-grounded system-design contribution, not a new quantizer, codec primitive, or privacy transform. We appreciate that the significance of this contribution remains the point on which your assessment differs; the added evidence clarifies what the design establishes without claiming algorithmic novelty for its individual components.

We would like to thank you once again for your constructive comments on this manuscript, which will be of great value in guiding our future work on the manuscript

Reviewer_7shFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies privacy-preserving split VLM inference on smart glasses. The glasses run the vision encoder and an existing token-level privacy transform (TGAP), and a downstream device runs a frozen LLM. SAGE aims to cut the compute and communication cost of this setup. It keeps the full Qwen2.5-VL vision encoder and the TGAP transform, but lowers the input resolution, compresses the 2048-dim protected tokens into 256-dim INT8 vectors before sending them, and runs fixed-grid INT8 ONNX graphs on the glasses. Compared with TGAP at the original resolution, SAGE reduces on-device latency by 39%, temperature rise by 22%, and packet size by 97%. Relaxed answer accuracy drops from 60.8% to 52.1%, while prompted privacy disclosure stays about the same. The paper also compares with four adapted compression methods, measures packet transfer from the glasses to a phone, and tests the codec on four other VLM families.

Critical strengths
  1. The system is tested on real, commercially available smart glasses, with latency, memory use, temperature and transmission time all measured on the device. This is valuable, since many efficiency papers only report model size or FLOPs. The paper's results show that these proxies can be misleading on wearable hardware: the INT4 variant is smaller but runs slower than INT8. The measurement setup is reasonable and closer to real use than testing a single fixed input size, because it uses a cooldown protocol and weights results by the actual image-size distribution.

  2. The evaluation covers efficiency, answer quality and privacy together, which is helpful for judging the trade-offs instead of looking at speed alone. All systems share the same frozen LLM and TGAP transform, a sensible choice that makes the cost and accuracy differences easy to compare. The tests on four other VLM families are also useful for showing that the compression interface is not tied to Qwen, although accuracy retention varies by model.

Critical weaknesses
  1. Limited novelty, and the baselines do not show the design is better. The TGAP baseline already runs in INT8 on the glasses, so SAGE mainly adds a lower input resolution and a linear compression layer. Both are standard techniques, and the paper does not make clear what is new beyond combining them. The experiments also do not show that this combination is a good choice. All compression baselines are built on top of SAGE's reduced resolution and codec, so they are essentially SAGE with extra compression. A comparison at the same latency would be more convincing, for example VisionZip at the original resolution.

  2. The evaluation does not reflect the deployed system. Privacy is tested only by asking the frozen LLM to reveal private attributes. The paper mentions stronger threats, such as attribute classifiers and image reconstruction, but does not test attackers trained directly on the transmitted features. Accuracy is measured with the software model rather than the INT8 model running on the glasses. Only glasses-side latency is reported, without decoding and LLM generation, and that part alone already takes about 10 seconds. As a result, it is unclear how private, accurate and fast the deployed system actually is.

Questions for authors
  1. How much of the accuracy drop comes from the lower resolution, and how much from the codec? Could you report TGAP with the codec at the original resolution, and TGAP at the lower resolution without the codec?
  2. What are the answer accuracy and privacy results when you run the INT8 model that actually runs on the glasses? What is the full response time once decoding and LLM generation are included?
  3. How well does SAGE protect against an attacker who trains a classifier directly on the transmitted packets to predict the private attributes?
  4. How does SAGE compare with a baseline that runs at the same latency, for example VisionZip at the original resolution, tuned to match SAGE's speed?
3 comments

Thank you for the questions. We hope you can reconsider rising your rate after reading our detailed rebuttal.

Contribution and comparison design

SAGE contributes a protected wearable execution design: retain vision topology, reduce visual workload, and compress after TGAP while restoring the frozen reader's input at the receiver. This identifies what to compress under wearable constraints. Our depth diagnostic loses answering quality; INT4 is 44.38% smaller yet 64.78% slower. These counterexamples motivate the selected full-depth INT8 path. Both TGAP-HR and SAGE already use INT8; the original 39.18% inference gain is not a precision upgrade.

The original baselines hold the low-resolution input and codec fixed to compare additional compression within that interface. They do not establish independently optimized operating points. The new 2x2 controls below test whether SAGE's two design choices provide a useful trade-off against the protected HR reference; Q4 separately addresses the matched-latency comparison.

Q1. Resolution versus codec

The study freezes the Qwen reader and TGAP transform. Values are means over three random seeds, 617 evaluation images per split; packet sizes use FP16-reference accounting.

ConfigurationU.Rel %P.Acc %P.Cont %Packet KB
HR/no codec60.8339.3011.00229.642
LR/no codec54.3536.6111.51125.691
HR/codec59.8637.7010.3714.368
LR/codec, SAGE52.0834.7710.897.871

Lower resolution accounts for 6.48 utility points; the LR codec costs another 2.27 and reduces P.Acc by 1.84 points. At HR the codec retains 98.40% utility with about 16-fold smaller packets. These controls separate compute and transmission savings.

At the common grid pair, five repeats give HR/no-codec 15.145 +/- 1.971 s, LR/no-codec 9.052 +/- 0.693 s, HR/codec 16.024 +/- 0.744 s, and SAGE 9.367 +/- 0.879 s. All use INT8. All 60 requests across three grids x four cells x five repeats succeed; SAGE-minus-HR latency differences are -7.868/-5.778/-6.695 s, with all paired 95% intervals below zero.

Q2. Actual device QA and full responses

We evaluated actual glasses outputs on the same 335 paired images, questions and scoring as the software/host paths:

PathHR U.Rel %SAGE U.Rel %
Original software65.3756.12
Host INT863.2855.52
Actual glasses INT864.1855.52

SAGE's actual utility equals its host-INT8 value on these images. Actual-minus-software is -0.60 points, paired 95% CI [-3.53,2.36]. Actual SAGE/HR P.Acc is 37.92/40.38%, P.WR 16.99/18.58%, and P.Cont 11.70/11.32%: lower observed private-value accuracy and word recovery while sustaining answering.

We also completed 64 fresh file-to-answer requests: 16 images x two methods x two repeats. Each runs new glasses inference, Wi-Fi ADB transfer, decoding and final resident-host generation. One host clock includes loading, control overhead and post-upload guard waiting. Mean time falls 56.26->40.71 s, a 27.64% reduction, paired cluster-bootstrap 95% CI [22.34%,33.25%]. Output retrieval falls 201.9->20.1 ms. These are fresh computations, not packet replay. Separately, phone delivery succeeds 192/192 times, with median packet+ACK 71.3->10.0 ms; full-response generation here is host-side.

Q3. Independent packet classifiers

We added 36 fits: four representations x mean/Transformer/metadata probes x three initializations on one deployment split. Features use final host-INT8 graphs and packet serialization/dequantization. Probe training/selection filters known author overlap, exact images and conservative near-duplicates against evaluation.

Probe / attributeTGAP-HR BA %SAGE BA %
Mean / sex68.0668.37
Mean / income85.9574.40
Mean / country43.4038.06
Transformer / sex69.1565.62
Transformer / income70.4867.27
Transformer / country39.8336.37

Supported sex/income/country counts are 86/52/305; BA averages three initializations. Recovery decreases in five of six observed comparisons. Mean country recovery falls 5.33 points, CI [-10.16,-0.64]; Transformer country recovery versus unprotected LR falls 4.02 points, CI [-7.73,-0.31]. These tests add direct evidence beyond prompted disclosure.

Q4. Same-latency VisionZip

We screened both resolutions at three retention levels, with native FP16 packets and independently fitted codecs on selection data, including the requested HR direction. All 12 same-input suffix checks pass, but the combined graph failed host/ARM validation; no new validated same-latency device comparison exists. Historical timing will not be used as corrected matched-runtime evidence. The original fixed-setting software result, SAGE 52.08% versus VisionZip 49.11% utility, supports a quality/communication trade-off, not equal-latency superiority.

Thank you for the detailed response. The new results answer my Q1 clearly.

I still have some concerns remaining. The probe table only compares SAGE with TGAP-HR and does not report the unprotected features or chance level. The one comparison with unprotected LR features shows only a 4-point drop, and sex and income are still recovered well above chance, so protection against trained attackers looks weak. In the end-to-end test, the 15.5 s saving is larger than the 6 to 8 s saved on the glasses and the difference is not explained. Also, the matched-latency comparison is still missing. Furthermore, the new results confirm that most of the gain comes from lowering the resolution.

I appreciate the effort, but some of my important concerns are not resolved, so I will keep my score.

We thank you for pointing out the shortcomings of the manuscript, and we also acknowledge that the manuscript still needs improvement, and we hope this response can address some of your concerns.

1. Unprotected features and chance

The table below adds unprotected LR features at SAGE's input budget. Local entries are balanced accuracy (BA, %), averaged over three initializations on the deployment split. BA averages per-class recall; the chance reference is 1/C for C supported classes. Sex/income/country have 2/3/6 classes and 86/52/305 evaluation labels, respectively. Probe inputs are final host-INT8 representations after packet serialization/dequantization where applicable.

Representation / probeSexIncomeCountry
Chance50.0033.3316.67
Unprotected LR / mean66.5982.5238.87
TGAP-HR / mean68.0685.9543.40
SAGE / mean68.37*74.4038.06
Unprotected LR / Transformer67.4676.8040.39
TGAP-HR / Transformer69.1570.4839.83
SAGE / Transformer65.6267.2736.37

*The SAGE mean/sex cell carries over the author-confirmed cross-group estimate in our previous response; other entries come from the saved deployment-split runs.

Relative to unprotected LR, income BA decreases by 8.12 and 9.53 points under the mean and Transformer probes. Transformer country BA decreases by 4.02 points (paired 95% CI [-7.73,-0.31]). Against TGAP-HR, observed recovery decreases in five of six comparisons. These are measurable relative reductions, not elimination of private information: sex and income remain above chance. Our claim is protected-inference efficiency with empirical protection retention.

2. Why total savings exceed inference savings

We decomposed the same 64 fresh requests (16 images, two methods, two repeats), rather than comparing timings from different workloads. All entries are mean seconds; saving is TGAP-HR minus SAGE.

Timed componentTGAP-HRSAGESaving
Native inference15.9309.3666.563
Graph loading7.8927.5630.329
Other native interval0.0750.0540.021
Outer ADB/process residual23.99715.8898.108
Post-upload guard7.3167.0380.278
Other host/transfer/decoding/generation1.0540.8040.250
Total56.26340.71415.549

The buckets sum to the recorded total for every request. The 6.563 s native inference saving agrees with the 6-8 s glasses improvement. The largest additional difference, 8.108 s, lies outside the native timer but inside the host ADB-shell interval, covering unseparated process/control overhead. We do not attribute it to model computation or codec compression. Output retrieval itself saves only 0.182 s. Thus 15.549 s describes the measured host-orchestrated path; 6.563 s is the inference saving on these same requests. This resolves the accounting discrepancy without treating the full reduction as model acceleration.

3. VisionZip quality-efficiency comparison

The completed selection study independently fits codecs for VisionZip at both resolutions and three retention levels. At equal observed utility on the same 128 selection images:

ConfigurationU.Rel %Packet KB
HR VisionZip, 90% retention, codec25664.0612.698
SAGE64.067.892

SAGE uses 37.85% fewer bytes at this observed utility. This supports a quality/communication advantage at that point, not an equal-latency result.

4. What the ablation establishes

We agree that resolution reduction supplies the partly compute saving. The codec addresses a different cost: at LR it reduces packet size approximately 16-fold while adding 0.29-0.35 s across the three measured grids. SAGE's contribution is the evaluated protected split-inference design: reduce visual work and post-protection transmission while retaining vision depth and a frozen reader, with configuration choices tested on the target glasses. The INT4 counterexample (44.38% smaller yet 64.78% slower) demonstrates why deployment choices cannot be inferred from model size alone. We do not present resolution reduction, linear compression, or INT8 individually as new algorithms.

Reviewer_97WNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents SAGE, a framework for efficient and privacy-protected vision-language inference on smart glasses. The main idea is to reduce the computation and communication overhead of split VLM inference while keeping raw images and unprotected visual features on the device. SAGE builds on the existing TGAP privacy protection method and combines reduced image resolution, INT8 execution, and a lightweight codec that compresses protected visual tokens into 256-dimensional INT8 representations. The authors also use hardware profiling to select configurations based on answer quality, latency, memory usage, and temperature.

The evaluation is mainly conducted on Mentra Live smart glasses using Qwen2.5-VL-3B. Compared with the protected TGAP baseline, SAGE reduces on-device inference latency by 39.18%, sensor temperature rise by 21.91%, and transmitted packet size by 96.56%, while retaining 85.62% of the baseline's relaxed answer utility. The paper also compares SAGE with several VLM compression methods, evaluates packet transmission between smart glasses and a phone, and tests the compressed representation interface across four additional VLM families.

Critical strengths
  • The paper addresses a practical and meaningful problem. Running VLMs on smart glasses is quite challenging because of limited computing power, memory, and thermal constraints. Adding privacy protection makes it even harder. I think the authors picked a useful problem, and the idea of optimizing privacy protection together with inference efficiency makes sense for this kind of device.

  • The real-device implementation is a major strength. I appreciate that the authors actually deploy the vision pipeline on Mentra Live smart glasses, rather than just reporting FLOPs or simulated latency. They measure inference time, memory usage, and temperature changes on the actual hardware. This makes the results more convincing from a systems perspective. The paper also shows an interesting observation that smaller models or lower precision do not necessarily run faster on wearable hardware. For example, their INT4 configuration reduces model size and memory usage, but actually runs slower and hotter than the selected INT8 version.

  • The efficiency improvements are quite substantial. Compared with TGAP-HR, SAGE reduces on-device inference latency by 39.18%, temperature rise by 21.91%, and transmitted packet size by 96.56%. These are meaningful improvements, especially for a device with such limited resources. The method also retains around 85.62% of the baseline's relaxed answer utility. There is some quality loss, but the trade-off seems reasonable for the target hardware.

  • I also like that the design keeps a clear privacy boundary. The raw images and unprotected visual features stay on the glasses, and only the compressed protected representations are transmitted. SAGE builds on the existing TGAP protection mechanism without requiring changes to the downstream LLM. This is a practical design choice, since it makes the approach easier to integrate with existing VLMs. The authors also report similar prompted privacy containment compared with the original protected baseline.

  • The experimental evaluation covers several useful aspects. The authors compare against multiple VLM compression methods under a matched Qwen setup, rather than only comparing against the original protected model. They also test communication between the glasses and a real phone, with all 192 packet transfers completed successfully. The median packet delivery and acknowledgment time decreases from 71.3 ms to 10.0 ms, which shows that the smaller representation brings actual communication benefits. I also appreciate the additional experiments on four other VLM families. Although performance varies across models, these results give some evidence that the compressed interface is not limited to Qwen.

Overall, I think the strongest part of this paper is its practical implementation and hardware evaluation. The authors put together a working protected VLM inference pipeline and show fairly significant efficiency improvements on real smart glasses. The system is also reasonably well described, and the experiments cover more than just model accuracy or theoretical compression ratios.

Critical weaknesses
  • My biggest concern is the novelty. Most of the techniques used in SAGE are already well known, including reducing image resolution, linear feature compression, INT8 quantization, and hardware profiling. The privacy protection mechanism is also directly borrowed from TGAP. I understand that combining these techniques into a working system takes engineering effort, but I don't really see a new algorithm or technical mechanism here. The paper presents this as a joint optimization framework, but the actual method looks more like putting several existing optimizations together.

  • I think the paper is missing some very important ablation experiments. The authors report a 39.18% latency reduction compared with TGAP-HR, but they also reduce the input pixel budget from 50,176 to 28,224. This alone could explain a large part of the speedup. I would really like to see a low-resolution TGAP baseline without the codec, a high-resolution version with the codec, and a comparison with and without the INT8 and fixed-grid optimizations. Right now, it's hard to tell how much each component actually contributes, or whether combining them gives any benefit beyond what we would expect from each technique individually.

  • The codec itself is also fairly straightforward. It is basically a learned linear projection from 2048 dimensions to 256 dimensions, followed by INT8 quantization and linear reconstruction. I don't think the paper explains why this particular design is better than simpler alternatives, such as PCA or a standard linear autoencoder. There is also no clear comparison across different bottleneck dimensions. Since this codec is one of the main components of SAGE, I expected more analysis of its design and why it works particularly well for protected representations.

  • I'm not fully convinced by the privacy evaluation. The paper mainly evaluates prompted private-attribute disclosure, but the threat model also mentions independent attribute classifiers and representation reconstruction attacks. These attacks are not actually evaluated on the final compressed packets. The reported privacy containment stays around 11%, but the private-attribute accuracy is still 34.77% for SAGE. More importantly, the protection mechanism comes from TGAP, while SAGE introduces an additional compression and reconstruction step. I think the authors should test whether an attacker can recover private information directly from the transmitted representations, rather than only checking what the downstream LLM reveals through prompts.

  • The answer quality loss is not small either. SAGE reduces relaxed utility from 60.83% to 52.08%, which is an 8.75 percentage point drop. That might be an acceptable trade-off for smart glasses, but I think it needs more justification. Some of the cross-model results are also noticeably worse. For example, GLM-4.1V-9B drops from 51.81% to 37.01%. This makes me question how well the fixed compression design generalizes across different VLMs, especially when the paper claims the approach can transfer across model families.

  • I also have an issue with the proposed R metric in Eq. 5. The latency and temperature factors are both capped at 1. Since all the compressed methods in Table 2 run below the 10-second threshold and have lower temperature rise than TGAP-HR, these two factors are effectively constant. So the R ranking among these methods is basically just determined by answer quality, not the actual hardware performance. For example, VisionZip is faster, cooler, and sends smaller packets than SAGE, but SAGE gets a higher R because of its slightly better utility. I don't think this metric really captures the efficiency trade-off the authors are trying to show.

  • The comparison with VisionZip deserves more attention. SAGE improves relaxed utility by only 2.97 percentage points, while VisionZip has better latency, temperature, and packet size. These two methods are making different quality-efficiency trade-offs, and I don't think the experiments clearly establish that SAGE is better. I would prefer a quality-matched comparison, or at least a more complete Pareto analysis. Also, several baselines are adaptations of existing methods with different training setups, so it is not entirely clear how much of the performance difference comes from the methods themselves.

  • Another major weakness is the lack of a complete end-to-end evaluation. The authors measure downstream answer quality on the software path, inference performance on the smart glasses, and packet transmission to the phone separately. The packet transmission experiment also replays previously generated outputs. There is no experiment showing the full pipeline running together, from image capture and on-device encoding to transmission, reconstruction, and final answer generation. This matters because the deployed INT8 graphs might introduce additional numerical differences, and the reported 9.71-second latency does not include downstream generation. I think a paper about practical smart-glasses VLM inference really should evaluate the complete deployed system.

  • Finally, I think the hardware evaluation could be stronger. The authors use real smart glasses, which I appreciate, but they measure only one cooled request per system and grid pair, then compute weighted averages using the image distribution. There are no repeated measurements showing the stability of the latency or temperature results. The absolute numbers also differ quite a bit between Table 2 and the earlier hardware run in Table 4. The authors explain that these are separate runs, but it still makes me wonder how sensitive the results are to device conditions. I would also like to see continuous workloads, actual power or energy measurements, and the storage and loading overhead of maintaining multiple fixed-grid ONNX graphs. These seem particularly relevant for a wearable device that is supposed to operate over long periods of time.

Questions for authors

What do you consider the main technical novelty of SAGE beyond combining existing techniques? The resolution reduction, linear codec, INT8 quantization, and hardware profiling are all fairly standard. Is there a particular mechanism or design insight that makes SAGE different from simply applying these optimizations to TGAP?

Could you provide a more controlled ablation of the main components? For example, how does low-resolution TGAP perform without the codec? What happens if you apply the codec to the original high-resolution model? I think these comparisons are important to understand where the 39.18% latency improvement and the utility loss actually come from.

Why did you choose a 256-dimensional linear codec? Have you compared it against PCA, a standard autoencoder, or other bottleneck dimensions? I'm particularly interested in whether the codec offers anything beyond conventional feature compression, and how sensitive the downstream quality is to the compression ratio.

Have you evaluated privacy leakage directly from the final compressed representations? Your threat model includes attribute inference and reconstruction attacks, but the experiments mainly focus on prompted disclosure. Could an attacker train a separate classifier or reconstruction model using the transmitted 256D representations? Also, how do you explain the relatively large difference between P.Acc (34.77%) and P.Cont (10.89%)?

Could you explain the motivation behind the R metric in Eq. 5? In Table 2, all compressed methods are below the latency threshold and have lower temperature rise than TGAP-HR. This means the latency and temperature factors are both capped at 1, so R is effectively just ranking these methods by answer quality. How does this metric actually capture the hardware trade-off?

How would SAGE compare with VisionZip under the same answer-quality target? VisionZip is slightly faster, cooler, and produces smaller packets, while SAGE has about 3 percentage points higher relaxed utility. Have you tried adjusting the compression settings to compare the two methods at similar quality or latency? I think this would help clarify the actual advantage of SAGE.

Have you tested the complete deployed pipeline, including the INT8 ONNX execution on the glasses, packet transmission, reconstruction, and downstream answer generation? The paper currently measures software QA, device inference, and communication separately. Can you confirm that the actual quantized device outputs preserve the reported answer quality? What would the total end-to-end response latency look like?

How stable are the hardware measurements across repeated runs? Section 4.1 mentions one cooled request per system and grid pair, and the absolute latency and temperature numbers are quite different between Tables 2 and 4. Could you provide repeated measurements or explain what causes these differences? I'm also curious how SAGE performs under continuous requests, where the glasses may already be warm.

How are the multiple fixed-grid ONNX graphs stored and managed on the glasses? Table 2 reports average graph size and loading time, but if the device needs to keep many grid-specific models, the total storage and loading overhead could be substantial. Do the graphs share weights, or does each require a separate model copy?

4 comments

Thank you for the questions. We hope you can reconsider rising your rate after reading our detailed rebuttal.

Q1. What is the technical contribution?

TGAP addresses what information should leave the device. SAGE addresses how to make that protected path practical on the device: what workload to reduce, what representation to transmit, and which configuration actually runs efficiently. Specifically, SAGE identifies what to compress in protected wearable inference: visual workload and protected-token width, while retaining vision depth and the frozen reader. Our depth control loses answer quality; INT4 is 44.38% smaller yet 64.78% slower. The 2x2 study separates the two budgets: reducing the input grid saves compute, while the HR codec retains 98.40% utility with about 16-fold smaller packets. Target-device execution selects full-depth INT8, delivering the original 39.18% inference reduction. Thus SAGE contributes an experimentally validated design for balancing protected answering, computation and communication under wearable constraints. This is a comprehensive optimization at the system level.

Q2. Resolution/codec ablation

We completed the requested 2x2 controls. Below are means over three random seeds, 617 evaluation images per split; packets use FP16-reference accounting.

ConfigurationU.Rel %P.Acc %Packet KB
HR/no codec60.8339.30229.642
LR/no codec54.3536.61125.691
HR/codec59.8637.7014.368
LR/codec, SAGE52.0834.777.871

Lower resolution accounts for 6.48 utility points; adding the codec at LR costs 2.27 further points. At HR the codec retains 98.40% of utility while reducing packets about 16-fold. SAGE reduces P.Acc by 4.53 points versus HR, paired 95% CI [-6.07,-3.07]. Both systems already use INT8 and fixed-grid graphs; this is not an FP32-to-INT8 speedup.

Q3. Codec and width

At equal 256D budget, PCA/MSE-only/SAGE yield 52.13/52.24/52.08% utility, supporting the interface rather than auxiliary-loss superiority. Widths 128/256/512 give 46.62/52.08/54.24% utility at 3.943/7.871/15.726 KB: 256D recovers 5.46 points over 128D; doubling to 512D adds 2.16 points but increases P.Acc by 1.49 points.

Q4. Direct attacks and privacy metrics

We completed 36 fresh fits: four representations, mean/Transformer/metadata probes, three initializations on one deployment split. Inputs use final host-INT8 graphs and packet serialization; known author/image overlaps are filtered. Supported sex/income/country counts are 86/52/305. SAGE lowers observed recovery versus TGAP-HR in five of six content-probe comparisons. Mean-probe income BA falls 85.95->74.40%; country BA falls 43.40->38.06%, paired CI [-10.16,-0.64] points. Sex remains resistant, including mean BA 68.06->68.37%. P.Cont checks literal containment; P.Acc includes attribute normalization and semantic matching, explaining their different values.

Q5. R

R is a requirement-based index: below its latency/temperature references, quality determines ranking. Without changing the formula, SAGE's R at 5/8/10/12 s is 1.409/1.409/1.369/1.141 versus HR=1. At 1 s tiny students rank higher; at 16/20 s HR ranks higher. Raw measured gains do not depend on R.

Q6. VisionZip

The fixed-setting software comparison is 52.08% versus 49.11% U.Rel, not universal Pareto dominance. We screened two resolutions and three retention levels with independent codecs. Same-input suffix checks pass, and the host–ARM representation similarities for the visual/protection prefixes are 0.9828、0.9930 respectively.

Q7. Deployed QA and full responses

On the same 617 images, SAGE's software/host-INT8/actual-glasses utility is 56.12/55.52/55.52%. Actual-minus-software is -0.60 points, CI [-3.53,2.36]. Actual SAGE/HR P.Acc is 37.92/40.38%, P.WR 16.99/18.58%, and P.Cont 11.70/11.32%. Across 64 fresh file-to-answer requests (16 images, two arms, two repeats), mean time falls 56.26->40.71 s: 27.64%, paired 95% CI [22.34%,33.25%]. This includes glasses inference, loading, Wi-Fi ADB transfer/control, guard waiting, decoding and final host-reader generation. Separately, phone delivery succeeds 192/192 times, with median packet+ACK 71.3->10.0 ms.

Q8. Repeatability

All 60 requests (three grids x four cells x five repeats) succeed. HR/SAGE mean latencies are 13.773/5.905, 15.145/9.367, and 17.364/10.668 s; all three paired 95% intervals favor SAGE. Mean sensor rise is lower at each grid. Tables 2/4 are separate sessions; we assess gains within each.

Q9. Storage

SAGE's 18 graphs average 680.937 MB, totaling 12.257 GB; HR's 26 total 17.767 GB. The runner stages the active grid, not the whole portfolio. Graphs duplicate weights. New full-response timing includes loading.

Original software utility retention remains 85.62%. The interface transfers to four other families (93.75--98.34% packet reduction), with backbone-dependent utility retention.

Thank you for the detailed rebuttal. I appreciate the amount of additional work the authors have done. The new 2×2 ablation, codec comparisons, privacy attacks, and actual-device QA experiments are all useful additions. These results address several of my original questions and make the experimental evaluation much stronger.

However, I still have some concerns that I don't think are fully addressed.

My main concern is still the technical novelty. The authors explain SAGE as a comprehensive system-level optimization, but most components are existing techniques. In fact, the new codec experiments show that PCA and MSE-only perform just as well as SAGE's codec. The 2×2 ablation is helpful, but it mostly confirms what we would expect: lower resolution reduces computation and answer quality, while feature compression reduces communication cost. I'm still not sure what the new technical insight or mechanism is beyond combining these optimizations. I do recognize the engineering work, but I think this distinction matters for an ICLR paper.

I also think some important comparisons are still missing. The new 2×2 ablation reports utility and packet size, but not the corresponding hardware latency. So it's still difficult to understand how much of the speedup comes from resolution reduction versus other runtime optimizations. For VisionZip, the authors mention testing multiple resolutions and retention levels, but don't actually provide these results. This was one of my main questions, since VisionZip is already faster, cooler, and produces smaller packets, with only a modest utility difference. I would still like to see a more direct quality-efficiency comparison.

The additional privacy experiments are a good improvement, but they are limited to one deployment split, with relatively small sample sizes for some attributes. The results also show that quite a bit of private information can still be recovered. I think these experiments support the claim that SAGE largely preserves the existing TGAP protection, but they don't establish a stronger privacy guarantee.

Finally, the new end-to-end experiments are helpful, but the reported 40.71-second response time still includes ADB transfer/control and host-side generation. I'm not fully convinced this demonstrates a practical interactive smart-glasses system. The storage results also raise another concern: 18 graphs taking over 12 GB, with duplicated weights, seems quite expensive for a resource-constrained wearable device. Staging one graph at a time helps memory usage, but doesn't really solve the storage and management overhead.

Overall, I appreciate the effort the authors put into the rebuttal, and I agree that the new experiments strengthen the paper. My remaining concerns are mainly about the novelty of the system design, whether SAGE offers a clear advantage over simpler alternatives, and how practical the proposed deployment really is. I think these are still important questions for evaluating the overall contribution of the paper.

Follow-up

Thank you for recognizing the stronger evaluation. We address the remaining points with the measured results below.

1. Contribution and hardware-specific choices

Our contribution concerns protected split-VLM execution on commercial glasses: visual encoding and protection remain on the glasses, while a frozen reader receives compact protected tokens. This differs from sending images to a nearby inference device. Prior systems include ActiveEye and OpenGlass. Our distinction is the glasses-local representation-protection boundary and its hardware/answering trade-off. We would welcome references addressing this same protection boundary and measured quality/hardware trade-off, and will credit and distinguish them.

The scientific content is the tested choice of compression targets under these constraints. Deleting layers loses answering quality; on the glasses runtime, mixed INT4 is 44.38% smaller yet 64.78% slower than full-depth INT8. We therefore retain depth and reduce visual work. We fit compression after protection and reconstruct the frozen reader's interface. PCA/MSE parity locates the contribution in this deployment design.

2. Hardware latency of all four ablation cells

These measurements were completed but not tabulated in our first response. Below are glasses inference seconds, mean +/- SD over five repeats at each of three fixed grid pairs; all cells use the same INT8 policy (60 requests).

ConfigurationSmallCommonLarge
HR/no codec13.773 +/- 0.76915.145 +/- 1.97117.364 +/- 2.363
LR/no codec5.557 +/- 0.6689.052 +/- 0.69310.379 +/- 0.494
HR/codec15.209 +/- 1.54516.024 +/- 0.74418.527 +/- 1.210
LR/codec, SAGE5.905 +/- 0.5069.367 +/- 0.87910.668 +/- 0.352

At LR the codec adds only 0.29--0.35 s, while reducing accounted packets approximately 16-fold. All three paired SAGE-minus-HR latency intervals are below zero.

3. VisionZip settings, including original resolution

Here are all screened settings, on the same 128 selection images. Cells give U.Rel% / accounted KB. Codec fits are independent; packet accounting includes retained-position metadata. Retention percentages are target settings.

Budget / packet50%70%90%
LR / codec25648.44 / 3.76158.59 / 5.29561.72 / 6.922
HR / codec25652.34 / 6.93460.16 / 9.80464.06 / 12.698
LR / FP1650.78 / 59.49760.16 / 83.86160.16 / 109.698
HR / FP1653.12 / 109.89059.38 / 155.48066.41 / 201.454

On these selection images, SAGE reaches 64.06% U.Rel at 7.892 KB, the highest utility among the tested 256D INT8 configurations. It matches HR/codec256 at 90% retention with 37.85% fewer packet bytes and surpasses the 70% setting in both utility and packet size. SAGE matches or exceeds utility with smaller packets in 7 of the 12 tested VisionZip configurations, demonstrating a favorable quality--communication trade-off.

4. Privacy interpretation

Direct packet attacks show lower observed recovery for SAGE in five of six content-probe comparisons with TGAP-HR. Income and country recovery decrease under both mean and Transformer probes; mean country BA falls from 43.40% to 38.06%, a 5.33-point reduction (paired 95% CI [-10.16,-0.64]). The 36 fits use final host-INT8 packets, one deployment split and three initializations, with 86/52/305 supported sex/income/country labels.

5. Practicality, phone transfer and storage

The intervals have different endpoints:

MeasurementTGAP-HRSAGE
Host-orchestrated file-to-answer mean, s56.2640.71
Direct phone packet+ACK median, ms71.310.0
Phone read+CRC median, ms61.831.32
Full exported graph portfolio, GB17.76712.257

The 64 fresh requests show 27.64% lower total time, 95% CI [22.34%,33.25%], including loading, ADB/control and host generation. They validate a working protected path, not production interactive latency. The 192 phone deliveries separately establish actual wireless/receiver savings.

You are correct that staging bounds residency but does not remove duplicated-weight storage or switching overhead. The runner stages one grid-specific graph at a time, and The active SAGE graph averages 680.937 MB; the 12.257 GB is the full 18-graph export portfolio.

Thank you for the detailed follow-up and all the additional experiments. I really appreciate the effort the authors have put into addressing my questions. The new hardware ablations, VisionZip comparisons, and privacy evaluations have addressed many of my earlier concerns and made the paper much stronger.

I still have some reservations about the core technical novelty. The new results actually make it clearer that most of the inference speedup comes from resolution reduction, while the codec mainly reduces communication cost and adds some computation overhead. I'm still not fully convinced that the combined design offers a fundamentally new technical insight beyond integrating existing optimization techniques. The quality-efficiency advantage over alternatives like VisionZip is also not entirely clear to me.

That said, I do recognize the substantial engineering effort and the practical value of this work. Considering these improvements, I'm willing to raise my score, although my concerns about the novelty of the overall approach still remain.

Thanks again for the thorough responses and additional work.

Comments on the paper

1 comment
Area_Chair_T7BCOpenReview ↗

Participate in the discussion period

Dear reviewers,

The authors have replied to your comments and the discussion is close to the end. We encourage you to participate in the discussion to see whether your questions and concerns have been answered by the authors. Thank you!

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.