- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes a novel caching algorithm that uses entailment instead of image similarity for text-to-image generation. While the problem and the idea are interesting, the solution is incomplete.
Critical strengths
S1. Timely and important problem.
Critical weaknesses
Thank you for submitting to ICLR. I think you are working on a very important problem and have a solid amount of work, but the solution is incomplete, unfortunately.
W1. The solution is incomplete.
a. The most important problem with the paper is that it does not describe how to perform entailment… The whole related work and motivation go to a deep understanding of image content beyond computing the trivial similarity score, which makes the problem very interesting. Yet, neither the main body of the paper nor the introduction has a single line on how entailment is done. Figure 2 has a large box for it, but the text does not describe how it is done. Entailment is really at the core of this paper, and it is a highly non-trivial problem to solve. I will talk more about this in my other comments.
b. Section 3.2, Equation 7, “With probability 1 − g(s), the returned image does not satisfy the request.” — this does not make sense. First, Figure 3 does not include any component that decides whether the output satisfies the request. Second, what does “satisfying” mean here? This is just not acceptable. Is there a human in the loop saying “this is OK”? The solution is clearly incomplete. Later in the experimental evaluation, we learn that a GPT is used for that, which really is not acceptable. If something is part of the solution, it has to be part of the solution, not part of the experimental evaluation. Evaluation is for evaluation. Plus, GPT is configured with a specific score of 50, for no clear reason…
Entailment is the place where the decision is made on whether the image is satisfactory or not. Using GPT afterward, unfortunately, shows that the entailment model is not successful. I will talk about this more in my other comments.
Also, what is C_rec here? C_tx and C_g are the recovery costs, aren’t they? What is the extra C_rec?
c. The entailment model ends up being one of the transformers: RoBERTa, DeBERTa, or ViLT. However, it is not clear how they decide whether there is entailment or contradiction… Section 3.3 has no single line on it, which makes the paper incomplete. I will talk about this more in my other comments.
d. OK, the core of the paper is unfortunately one single paragraph in the experimental setup (Models and judgements)… This is not OK, unfortunately. The body of the paper should cover it in detail.
First, there should not be judgments in your evaluation, as judgment is the entailment itself. Why do we have another judgment once the entailment is done? Figure 2, for example, does not have this in its plot… Why not? Motivation also does not talk about it.
Second, entailment uses three categories: entailment, neutral, and contradiction. Figure 2 has two… What is neutral, what to do when it is neutral, and why have it?
Third, deciding entailment when entailment scores the highest is really not a strong metric. Highest might mean anything… It might be just a little above contradiction… This weak entailment model is probably the reason that you need a judge at the end.
W2. There are unclear points.
a. Section 3.2: “C_em, the inference cost of model m” — which model? I reverse-engineer it to be the “entailment model”; it could have been other models, as your pipeline has multiple models. Please specify clearly.
Having said that, “entailment model” is also not clear, as there is only NLI inside the figure, and there is no single line on what an entailment model is. It can be anything. It can be a sophisticated model doing multiple things, etc. NLI does not mean entailment.
b. Image transmission cost C_tx is not clear. What is the end-to-end workflow here? Is it the cost to send the image from the machine to the user, over the network? How do we calculate the network cost? How much bandwidth do you assume? Or is everything running locally?
c. Eq. 7 has C_tx, C_g, and C_rec for failing cases, but Eq. 8 does not have them for (1-e_m(s)). It only has C_g. Why?
d. What are R_G and R_D? Why do we need them? You already defined the costs in the previous sections, haven’t you?
e. The paragraph after line 245 is a repetition. We understand by now how thresholds work. We need to know how you chose it. The paragraph from 253 explains how they are chosen, but would benefit from an expanded description, with more detail on the reasoning behind the cost optimizations, perhaps with some examples.
f. Also, Figure 3 uses a specific dataset, doesn’t it? Which one is it? Isn’t it a limitation/requirement of your method? What if we don’t know the dataset ahead? This is another good item for the limitations and discussion section.
g. Another issue of the paper is that, at the high level, it contradicts its own problem statement. The problem statement refers to the fact that using only similarity is not enough, as it can eliminate images with low similarity scores. Whereas the solution nevertheless relies only on the top-1 similarity score to decide whether an image should be evaluated in the first place. Top-K candidates could, for example, alleviate this problem. I understand that after getting the top-1 candidate, there are more evaluations in the system; nevertheless, if the similarity score is insufficient, the second candidate might actually be a better fit for the query… This could be a good item for a “discussion and limitations” section. Or, potentially, be incorporated into a future version of the solution.
W3. The presentation can be improved.
a. What is directional semantic coverage? Is this a technical term? If so, can you please cite it? If not, the word “directional” is ambiguous.
b. The right-hand side image of Figure 1 might be disturbing to some people, as it might imply having a violence in it. Let’s please replace it with another one conveying the same message.
c. Contributions (last item of it, generally) should provide, quantitatively, how much improvement the algorithm provides in terms of accuracy/cost.
Questions for authors
Q1. Why do you have a judge at the end of your evaluation, and why is it not in Figure 2?
Q2. Why does your Figure 2 not include "neutral" in your entailment box? What is neutral, and what should we do when it is neutral?
Q3. What is an image transmission cost?
4 comments
continue from the previous response
Image transmission cost.
The transmission cost represents the latency associated with delivering an image from the service provider to the user. In our experiments, this is modeled as a fixed service-latency parameter and applied consistently across all caching policies.
Generation and recovery costs.
Our objective is to minimize the total time required for a user to obtain a satisfactory image. Image generation cost represents the time required to produce a new image. Recovery cost represents the additional time incurred when a user receives an unsatisfactory cached image, recognizes the failure, and submits another request. This user-side re-prompting delay is separate from the time required to generate and transmit the replacement image. All cost components are expressed in seconds.
Therefore, an unsuccessful cache return incurs the initial transmission time, followed by the additional recovery time and the generation and transmission times required to deliver a satisfactory replacement. These components are explicitly included in our direct-reuse expected-cost formulation.
Why verification rejection incurs only generation cost.
When an entailment verifier rejects a cached candidate, the cached image is never transmitted to the user. Instead, the system immediately generates and transmits a new image. Since the verification cost function already includes the transmission cost for the image eventually delivered, only the additional generation cost is required in the verifier-rejection term.
In contrast, when a verifier incorrectly accepts an unsatisfactory cached image, the system incurs additional regeneration, retransmission, and recovery time.
Expected-cost functions.
The generation, direct-reuse, and verification cost functions represent three alternative actions available to the caching system. Their purpose is to compare the expected total times of these alternatives at a given similarity score, allowing EACache to determine whether a cached image should be returned directly, verified, or replaced through generation.
We have revised the cost-model description to explicitly distinguish the meanings of generation, transmission, and recovery costs, and explain why the different actions incur different cost components.
W2(e, f): Threshold Selection and Dataset Dependence
We thank the reviewer for requesting additional details regarding the threshold-selection procedure.
Threshold estimation.
We estimate the conditional probabilities using labeled image–query pairs from the threshold-learning dataset. The pairs are grouped into adaptive similarity bins, where we calculate the observed satisfaction rate, verifier acceptance rate, and satisfaction rate among verifier-accepted pairs. We then fit smooth, monotonically increasing logistic probability curves using the observed success and failure counts, giving greater influence to bins containing more observations.
The fitted probability curves are incorporated into our expected-cost model to determine the decision boundaries. The lower threshold is obtained where generation and verification have equal expected costs, while the upper threshold is obtained where verification and direct reuse have equal expected costs. All thresholds are learned exclusively from threshold-learning data and remain fixed during held-out evaluation.
Dataset dependence and Figure 3.
We clarify that Figure 3 is based on the SNLI-VE/Flickr30K dataset, illustrating the expected-cost curves and learned thresholds for the three entailment models under that experimental setting.
The thresholds are data-dependent because the relationships among similarity, verifier acceptance, and user satisfaction can vary across datasets. Accordingly, we independently estimate the probability functions and decision thresholds for each evaluated dataset rather than transferring the thresholds learned on SNLI-VE/Flickr30K to COCO or DiffusionDB.
Importantly, EACache does not require prior knowledge of future incoming queries or the entire deployment dataset. It requires only a representative calibration sample to estimate the probability functions and determine the thresholds before deployment.
If the query or cache distribution changes substantially, the same calibration procedure can be repeated using more recent representative data. We have clarified this requirement and explicitly identified the dataset used in Figure 3 in the revised manuscript.
continue from the previous response
W2(g): Top-1 Retrieval, the Role of Similarity, and Extension to Top-K
We agree with the reviewer that restricting retrieval to the highest-similarity candidate may overlook valid images with lower similarity scores.
However, we would like to emphasize that although similarity alone cannot reliably determine whether a cached image satisfies a user query, it remains a highly informative and computationally efficient signal. Our motivation is not to replace similarity-based retrieval with entailment verification, but to combine their complementary strengths.
Similarity allows EACache to efficiently narrow the candidate space and distinguish cases where direct reuse or generation is preferable without invoking an entailment model. Specifically, high-similarity candidates can be returned directly, while sufficiently low-similarity candidates trigger generation. Only candidates in the intermediate similarity region require entailment verification, substantially reducing the need to perform expensive semantic verification on every cached image.
Our current framework uses top-1 retrieval to isolate the benefits of selective entailment verification and maintain a tractable cost-aware decision policy. Nevertheless, we acknowledge that the highest-similarity candidate is not necessarily the most semantically suitable image, and a lower-ranked candidate may better satisfy the query.
Extending EACache to top-K retrieval could further improve cache-reuse opportunities by allowing additional candidates to undergo selective verification. Such an extension would require accounting for additional verification costs and determining when to stop examining candidates and invoke generation. We consider this an important direction for future work and have explicitly acknowledged the top-1 restriction in the manuscript.
W3(a): Clarification of Directional Semantic Coverage
We thank the reviewer for highlighting this terminology issue.
By directional semantic coverage, we mean that the cached image must provide sufficient visual evidence to support the content requested by the incoming query, while allowing the image to contain additional details not specified in the query.
For example, an image depicting two people standing beside several vehicles may satisfy the query "people near cars," even though the image contains additional objects and visual details. The reverse implication does not necessarily hold.
This asymmetric relationship motivates using visual entailment rather than relying exclusively on symmetric image–text similarity. Our formulation follows the premise–hypothesis interpretation of visual entailment, where the image provides the evidence and the user query specifies the content that must be supported.
We have clarified this definition and its relationship to existing visual-entailment formulations.
W3(b): Figure 1 and Ethical Concerns
We appreciate the reviewer for bringing this issue to our attention. We agree that the original example in Figure 1 may be interpreted as depicting a potentially disturbing situation.
We have replaced the right-hand example with a neutral image–query pair that illustrates the same distinction between semantic entailment and image–text similarity, without potentially disturbing visual content.
W3(c): Quantitative Contributions
We agree that the contributions should more clearly communicate the quantitative benefits of EACache.
In response to the concerns about the original similarity-only baseline, we introduced a stronger cost-calibrated baseline using the same expected-cost formulation as EACache.
Under this comparison, EACache reduces realized mean system cost by 5.64–8.10% on SNLI-VE/Flickr30K and 0.67–2.28% on COCO, depending on the selected entailment model.
We additionally evaluated EACache using AI-generated images from DiffusionDB with automatically generated Qwen3-VL-8B captions and generalized incoming queries derived from the original DiffusionDB generation prompts. Under this setting, EACache achieves 0.85–1.51% lower mean system cost than the similarity-only baseline at .
These additional experiments provide a more rigorous assessment of the framework and demonstrate that its cost advantage persists under the evaluated stronger baselines and generated-image settings. We have revised the contribution statements to incorporate these quantitative findings while acknowledging the smaller improvements under stronger comparisons.
continue from the previous response
Q1: Why Is There a Judge at the End of the Evaluation, and Why Is It Not Included in Figure 2?
GPT-5.2 is used only in our experiments as a proxy for user satisfaction. Its purpose is to determine whether cached images satisfy their corresponding queries, allowing us to obtain satisfaction labels for threshold learning and experimental evaluation.
In real deployment, actual user feedback determines whether a cached image is satisfactory. When a returned cached image fails to satisfy the user, recovery is triggered.
Figure 2 illustrates the deployed EACache workflow, in which similarity retrieval and selective entailment verification determine whether to return a cached image or generate a new one. User feedback determines satisfaction after a cached image is returned.
Since GPT-5.2 is only used in our experiments and is not invoked during actual online query processing, it is not included in Figure 2. We have revised the figure and its caption to make this distinction explicit.
Q2: Why Does Figure 2 Not Include Neutral in the Entailment Box? What Happens When the Prediction Is Neutral?
The entailment models predict three classes: entailment, neutral, and contradiction. Neutral indicates that the available evidence is insufficient to establish either that the query is supported or that it is contradicted by the image.
For cache reuse, we require positive evidence that the retrieved image satisfies the request. Therefore, a candidate is accepted only when entailment is the highest-scoring class. Both neutral and contradiction lead to rejection, after which the system generates a new image.
We have revised Figure 2 to explicitly show the neutral outcome alongside entailment and contradiction.
Q3: What Is an Image Transmission Cost?
Image transmission cost refers to the time required to deliver an image from the service provider to the user over the network. It is distinct from image-generation time and entailment-verification time.
In our evaluation, transmission is modeled as a fixed latency of 2 seconds per delivered image, applied consistently across all policies. If an unsatisfactory cached image is returned, the system incurs transmission time again when delivering the regenerated image.
Summary
We thank the reviewer for the detailed feedback. We have clarified the entailment-verification mechanism, the distinction between experimental satisfaction judgments and actual user feedback in deployment, system-cost accounting, threshold estimation, and the limitations of top-1 retrieval. Together with the revised figures, stronger similarity-only baselines, and additional AI-generated-image experiments, these revisions strengthen the evaluation and presentation of EACache.
Response to Reviewer aowg
We thank the reviewer for the detailed feedback and for recognizing the importance of the problem addressed in our work. We acknowledge that several aspects of the entailment verification mechanism, satisfaction evaluation, and cost-aware decision process require clearer explanations. We address these concerns below and clarify the intended scope of EACache.
W1(a, c, d) & W2(a): Entailment Verification Mechanism
We appreciate the reviewer's request for a more detailed explanation of how entailment verification is performed. We would like to clarify that EACache does not aim to train a new entailment model; rather, it introduces a cost-aware framework that determines when pretrained entailment models should be invoked for cache-reuse decisions.
Specifically, we evaluate three pretrained entailment models: RoBERTa, DeBERTa, and ViLT. For RoBERTa and DeBERTa, a vision-language model first generates a detailed caption for each cached image during offline cache construction. The cached caption serves as the premise, and the incoming user query serves as the hypothesis. The entailment model then determines whether the caption semantically supports the requested content. ViLT, in contrast, directly evaluates the image and query without requiring the intermediate caption.
Each verifier predicts three classes: entailment, neutral, and contradiction. We follow a standard three-class NLI inference procedure, where the highest-scoring class determines the prediction. A cached image is accepted only when entailment receives the highest score. Both neutral and contradiction are treated as rejection because neither provides sufficient evidence that the cached image satisfies the query. When verification rejects a candidate, EACache generates a new image.
The inference cost of verifier m, denoted by Ce,m, refers specifically to the computational overhead of the selected entailment model. Each model has its own measured inference latency, which is incorporated into the corresponding expected-cost function and threshold derivation.
We acknowledge that the original manuscript did not provide sufficient detail about these inference procedures. We have expanded the system description to explicitly explain the verifier inputs, three-class predictions, acceptance criterion, and respective roles of the textual and multimodal verification models.
W1(b, d) & Q1: Distinction Between Entailment Verification and Satisfaction Evaluation
We thank the reviewer for raising this important concern. We clarify that entailment verification and satisfaction evaluation serve two fundamentally different roles in our framework.
Entailment verification is part of online inference.
EACache uses RoBERTa, DeBERTa, or ViLT to determine whether a retrieved image should be returned to the user. These models provide predictions about semantic entailment but are not assumed to be perfectly accurate.
Satisfaction evaluation in experiments and real deployment.
In real deployment, satisfaction is determined by actual user feedback after receiving a cached image. In our experiments, we use GPT-5.2 as a proxy for user satisfaction to assess whether an image satisfies its corresponding query. These experimental judgments provide the labels required for estimating the probability of successful cache reuse and evaluating the resulting caching decisions.
The distinction is particularly important because a verifier can make incorrect predictions. For example, a verifier may classify an image–query pair as entailed even though the image does not fully satisfy the query. Conversely, it may reject a valid cached image. The independent satisfaction evaluation in our experiments allows us to quantify these errors and account for their consequences in the cost model.
Importantly, GPT-5.2 is used only in our experiments and is not part of the deployed EACache system. It does not determine online cache-reuse decisions or introduce additional online inference overhead. In real deployment, user feedback provides the satisfaction signal.
We selected GPT-5.2 after the authors manually examined its image–query satisfaction judgments and found them consistent with their own assessments. Since our experiments involve more than 10,000 image–query pairs, we use GPT-5.2 as a scalable proxy for manual satisfaction evaluation. We acknowledge that this constitutes qualitative author validation rather than a formally quantified human-agreement study.
We have clarified this separation in the manuscript and revised Figure 2 to distinguish online verification from user-feedback-based satisfaction evaluation.
W1(b) & W2(b, c, d) & Q3: Interpretation of the System Cost Model
We appreciate the opportunity to clarify the different cost components and their roles in EACache.