- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work addresses image processing chain forensics: given two versions of the same image (before and after), several input-output sample pairs for each unnamed operation, and a set of candidate processing histories, the task is to rank these histories. ChainPrint fits executable functions for each operation using a predefined model library (parameterized standard operators plus eight SD-1.5 inpainting configurations), chains them together according to the candidate histories, and ranks the results based on the final Mean Squared Error (MSE). Top-1 accuracy over eight steps is 100% for standard processing chains and 77.8% for chains involving AI-based editing.
Critical strengths
- The task is clearly defined, and a single framework covers various types of forensic problems—specifically those involving sequence, repetition, and partial spans.
- The reporting is transparent. Failure counts are tallied, pairwise confidence intervals are provided, and the data in the supplementary CSV files aligns perfectly with Tables 1–4.
Critical weaknesses
-
My primary concern is that the model library already encompasses the exact processes used to generate the data. The AI editor's prior corresponds precisely to the eight configurations sampled by the test editor (App. D, App. I); the paper itself acknowledges that it "does not discover an unknown editor." Regarding standard operators, the search spaces for JPEG and resampling cover the exact Pillow settings used during generation (App. F, App. I). While generic filters are used for blurring and sharpening—and the out-of-library NumPy experiment (Fig. 3c, 99.4%) offers some support—this applies only to standard operators. Thus, the 100% success rate is unsurprising. The 77.8% figure also aligns closely with the proportion of renderable candidates (140/180), given that a complete ranking implies correctness ("correct whenever complete," Sec. 5.2).
-
I fail to see a significant methodological contribution in the comparative experiments. Several modified baselines performed worse than random chance (1/16) even after up to 420 updates, which is inconclusive. The only direct comparison—using the same prior for joint fitting—showed essentially equal performance (90.32 vs. 89.68, with all confidence intervals including zero). In Table 3, correction, in-sample selection, and refinement shifted the Top-1 accuracy by at most 0.8 percentage points, with confidence intervals again spanning zero. Significant performance drops occurred only when replacing sequential composition with an additive model (−41.4) or shuffling pairings—both of which serve as weak baselines. The remaining benefit is a 5.3× reduction in preparation time, measured across six sources.
-
There is a fundamental disconnect between the theory and the experiments. The paper states that the exact invariance condition (Prop. B.2) does not assume validity for JPEG or AI editing operators, nor does the estimator compute stability or recovery certificates (Sec. 4.3). The supporting evidence relies on a 2D comparison, consistency across a maximum of nine standard model sets, and four configurations of a single editing operation.
Questions for authors
- How does ChainPrint behave if the AI edit falls outside the defined prior—such as with an unseen prompt, mask, or seed, or when switching to a different inpainting model?
- In the two-step process, errors persist even when the ranking is complete (FiveK: 164/240 complete, Top-1 67.1%). What is the source of these errors? Additionally, why did the eight-step FiveK process only complete 34/60 cases and generate just 58 queries?
- Aside from setup time and the small-scale 12-source AI Recall@5 diagnostic (91.7 vs. 66.7), are there other scenarios where ChainPrint outperforms joint fitting?
6 comments
Response to VFYT (2/4): Ranking Errors and Execution Availability (W1, Q2)
W1 (excerpt): The 77.8% figure also aligns closely with the proportion of renderable candidates (140/180), given that a complete ranking implies correctness ("correct whenever complete," Sec. 5.2).
Q2 (summary): What causes errors among complete two-step rankings, and why does FiveK produce only 58 eight-step targets and 34 complete rankings out of 60 planned queries?
Thank you for raising this issue. Your reading of the eight-step 77.8% result is correct. Of 180 planned queries, 173 produced targets and 140 yielded complete rankings, all correct. In this experiment, 77.8% therefore also equals the complete-ranking rate. We have traced the errors among complete two-step rankings to operator estimation and corrected them.
Errors among complete rankings (Q2)
FiveK's 67.1% is 161/240 overall, with three errors among its 164 complete rankings. We traced every error among complete two-step rankings, three from FiveK and one from COCO. Each reversed blur→AI into AI→blur, although the selected AI configuration was correct. Small blur-approximation errors changed the order comparison after the subsequent AI edit.
We therefore improved parameter fitting. The operator library and eight anonymous support pairs per operation stayed unchanged. We searched parameters using the existing models' hard-quantized forward passes. Adoption depended only on content-excluded support error. For the relevant operation, held-support MSE8 fell from 0.109775 to 0.0001443.
Table R2a. Original versus improved fitting, with the same query targets, candidates, MSE and complete-ranking criterion within each cohort; K=8 throughout. All counts retain planned queries.
| Two-step cohort | Fitter | Targets generated | Complete rankings | Correct / planned |
|---|---|---|---|---|
| Original 720 queries | Original | 697/720 | 604/720 | 600/720 |
| Original 720 queries | Improved | 697/720 | 608/720 | 608/720 |
| FiveK 240-query subset | Original | 225/240 | 164/240 | 161/240 |
| FiveK 240-query subset | Improved | 225/240 | 164/240 | 164/240 |
| Separate new 24-source SD1.5 cohort | Original | 23/24 | 19/24 | 18/24 |
| Separate new 24-source SD1.5 cohort | Improved | 23/24 | 19/24 | 19/24 |
Table R2a shows 600/720→608/720, with all four original complete-ranking errors repaired and no regression among the original 600 correct decisions. Another four queries, all conditions of one COCO source, gained complete, correct rankings. The 720 queries comprise 180 sources with four conditions each; FiveK is a subset, not another independent test.
The new 24-source test used a fixed blur/SD1.5 configuration, distinct from the SD1.4 cohorts in Response 1. Models were frozen before target generation. The improved fit corrected another instance of the same order confusion, again without reversing any previously correct decisions.
FiveK's 58 eight-step targets and 34 complete rankings (Q2)
Table R2b. Original eight-step availability and recovery. The FiveK row is included in the all-dataset row; these results do not use the new fitting.
| Cohort | Planned | Targets generated | Complete rankings | Correct / planned |
|---|---|---|---|---|
| All datasets | 180 | 173 | 140 | 140/180 |
| FiveK | 60 | 58 | 34 | 34/60 |
For FiveK, two of 60 target generations were refused by the editor's checker, leaving 58. Among these, 24 queries lacked candidate scores following execution refusals, leaving 34 complete, correct rankings (Table R2b). We will report all three counts separately so both failure points are visible.
For missing scores, we also added a conservative two-step decision rule. Errors on known pixels outside the mask lower-bound a missing candidate's MSE. A winner is returned only when these bounds exclude the missing candidates.
Table R2c. Different decision rules on the same original 720 two-step queries, using the improved estimates.
| Rule | Decisions / 720 | Correct histories / 720 |
|---|---|---|
| Require complete rankings | 608/720 | 608/720 |
| Conservative lower-bound rule | 680/720 | 680/720 |
| Rank available candidates only | 697/720 | 697/720 |
The conservative rule yielded 680/720 decisions, all correct, while direct available-only ranking yielded 697/720 correct histories (Table R2c). The bounds show why missing candidates cannot match or beat the winner under the fitted models. We will report these partial decisions separately from complete rankings.
Response to VFYT (3/4): Same-Prior Joint Comparisons and Module Contributions (W2, Q3)
W2 (summary): The learned baselines provide weak evidence, same-prior Joint performs similarly, and the selection, correction and refinement gains in Table 3 are small, with intervals including zero.
Q3 (summary): Beyond preparation time and the small 12-source AI Recall@5 diagnostic, does ChainPrint outperform Joint elsewhere?
Thank you for identifying the gaps in our strong-baseline comparisons and module evidence. The added experiments give two direct answers: ChainPrint outperforms Joint with matched native adaptation on real platform chains (Q3), and refinement and correction improve history-recovery accuracy under implementation mismatch (W2).
Same-prior Joint comparisons (W2, Q3)
We used public historical chains produced by actual platform uploads and downloads, fixing 52 query sources from 26 cameras absent from the support set. The method and split were frozen before query acquisition.
Table R3a. Matched native adaptation (P2), same 52 sources. Both methods share supports, prior budget, native dimensions and geometry, candidates and MSE. Each uses three separate K=8, G=1 cold fits, then assembles the operator library.
| Method | Two-step accuracy | Three-step accuracy |
|---|---|---|
| Native-adapted Joint | 41.45% | 13.46% |
| Native-adapted ChainPrint | 47.44% | 16.74% |
| CP−Joint, pp (95% interval) | +5.98 [2.56, 9.19] | +3.28 [1.85, 4.84] |
Table R3a shows gains of 5.98 and 3.28 percentage points. Intervals are clustered by the 26 cameras and correspond to prespecified secondary comparisons without multiplicity correction. This extends the evidence for Q3 to real-platform Top-1 recovery, with both methods receiving the same information and prior budget.
The advantage concerns this native-adaptation protocol, not every possible native Joint design. The original pixel version did not outperform Joint, and the JPEG and cross-checkpoint comparisons did not establish a consistent advantage. We will retain these results.
Roles of refinement and correction (W2)
Your observation about the small gains in the original Table 3 is correct. To test whether the modules improve history decisions, we fixed supports, prior, selection procedure, queries and candidates under FFmpeg JPEG implementation mismatch, changing only refinement and correction.
Table R3b. Same refreshed FFmpeg support set (8 pairs/operation), original function library, 180 eight-step queries, 16 candidates and MSE. Only the two module switches change.
| Refinement | Correction | Correct / 180 |
|---|---|---|
| Off (initial estimates) | Off | 139/180 |
| Off (initial estimates) | On | 139/180 |
| On | Off | 161/180 |
| On | On | 172/180 |
The full configuration recovered 172/180 histories, a net 33 more than no-refinement (+18.33 percentage points) and 11 more than no-correction (+6.11 points), as shown in Table R3b. Source-paired 95% intervals were [12.78, 23.89] and [2.78, 9.44]. All gains came from JPEG-containing histories; the 24 JPEG-free histories remained correct.
The ablation also reveals an interaction in this condition: correction gives no gain on the initial estimates, but improves accuracy after refinement. We will report the original small-gain results alongside this mismatch experiment.
Preparation cost was also independently checked.
Table R3c. Separate seven-condition conventional-processing experiment, original frozen fitters, same host and four CPU cores. Preparation CPU includes common AI-cache construction.
| Cost comparison | Observation |
|---|---|
| Joint/CP preparation CPU ratio | 5.66–5.90 |
| Conditions with slower CP queries | 5/7 |
The 5.66–5.90 Joint/CP preparation CPU ratio in Table R3c concerns building an operator library from new anonymous examples. It does not apply to the new parameter search in Response 2 or the P2 adaptation above. Query costs will be listed separately, since ChainPrint queries were slower in five conditions.
We agree that the below-chance learned-baseline results do not establish an advantage for our estimator. We will make same-prior Joint and these controlled ablations the principal experimental evidence, with learned baselines retained as supplementary results.
Response to VFYT (1/4) — Library Coverage and Unseen AI Edits (W1, Q1)
Thank you for raising this concern. The original benchmark does cover its generation settings. The prior supplies a range of functions to fit, and the anonymous pairs supply the observed behavior of each operation. Our view is that history recovery needs an approximation that preserves differences between candidate histories. We added experiments with generating implementations absent from the library to test this view.
W1 · A new implementation under the fixed library
We replaced Pillow JPEG with FFmpeg's native MJPEG encoder, keeping the function library fixed. Both implementation and operating points changed; this was not a quality-matched encoder comparison. Refitting with eight anonymous pairs per operation raised recovery from 128/180 to 172/180 on the same FFmpeg queries (Table R1a). The examples were separate from the queries.
Table R1a. Fixed original library, including Pillow JPEG but no FFmpeg implementation. Both rows use the same 180 FFmpeg eight-step queries, 16 candidates and MSE. Only the support set changes.
All 44 net gains came from JPEG-containing histories. Generator names describe the experiment; fitting received anonymous image pairs, not operation-type or encoder labels.
Q1 · Unseen AI configurations and editors
Refreshing examples was insufficient for the original eight fixed AI configurations (Table R1b). Complete rankings could still select the wrong history.
Table R1b. Original prior8 (eight fixed SD1.5 AI configurations) under four-step mismatch, with 16 candidates and MSE. K is the number of accepted support pairs per operation actually used.
The first two studies selected K=6 after checker refusals, using different support subsets. Their query sources overlap. Prompt and mask changed jointly; these rows do not isolate each factor or compare shift severity.
For SD1.4, we therefore extended the fitted-function family. The estimator was not given the SD1.4 implementation; it learned a pixelwise RGB affine approximation from the anonymous pairs. We froze the approximation before generating the 24 additional two-step targets. With the same eight SD1.4 support pairs, candidates and MSE, recovery rose from 5/24 to 14/24 (Table R1c).
Table R1c. Same 24-source SD1.4 two-step cohort, K=8 and 16 candidates, pixel MSE; the other three fitted operations stay fixed. SD1.4 generates examples and targets but is absent from the estimator.
On the 17 common-complete queries, correct counts changed from 5 to 9. Five of the six newly complete rankings were correct.
We also tested a constrained, shared-slope RGB affine map at each pixel, paired with local directional power scoring. After freezing both, we generated four-step queries from a separate 24-source cohort, disjoint from earlier mismatch queries and supports. Each true history called the same AI operation twice.
Table R1d. Separate SD1.4 four-step cohort: frozen shared-slope family, 16 candidates, all 24 targets and rankings complete. No original-prior8 comparison was run on this cohort.
With eight pairs, the extension recovered 12/24 histories (Table R1d). In the last support setting, changing the score gave +33.33 percentage points (paired 95% interval [8.33, 58.33]). These results support learning an approximation for history recovery when the generating editor is absent from the available functions. We will add the extension and its evaluation alongside the original-prior mismatch results.