acceptodds
Under review as a conference paper at ICLR 2027

Who Influenced This Behavior? Auditing Federated LLMs Without Revealing Client Updates

Abstract

Federated parameter-efficient fine-tuning (PEFT) can adapt language models to decentralized data while secure aggregation hides individual dense numerical learning updates. This confidentiality creates a post-hoc accountability problem: given an observed model behavior, how can an auditor rank the clients whose updates locally supported or opposed it? We introduce AudiTrace, a behavior-conditioned client-ranking method based on magnitude-free audit records. At selected checkpoints, each client locally records only the coordinate indices and signs of its Top- trainable-update entries; the dense update can remain on an independent secure-aggregation learning path. Given a differentiable audit objective, the auditor constructs an analogous query-gradient record and aggregates FedAvg-weighted signed overlap across checkpoints, yielding a directional ranking without reconstructing individual dense updates. We characterize exact mask-space geometry, derive conditional approximation bounds, and formalize a hidden-tail failure mode. Across diverse model, task, and partition settings, AudiTrace reaches dense-reference Spearman correlation –. Against exact leave-one-client-out retraining it attains Spearman and positive-effect AUROC from coordinate identities and signs alone, within Spearman of dense cosine. A signature retains Spearman with a 415.4 KiB audit record; repeated-sign membership inference peaks at AUROC versus chance. AudiTrace thus enables post-hoc auditing in federated PEFT while leaving individual dense updates on the secure-aggregation path.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
federated learning, large language models, parameter-efficient fine-tuning, client attribution, secure aggregation, behavior auditing, influence estimation, privacy-preserving machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nYzpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents AUDITRACE, a method for attributing a specified model behavior to clients in federated PEFT. Each client records the coordinate indices and signs of its largest updates, while dense numerical updates can remain on a separate secure-aggregation path. The auditor compares these sparse records with signed gradients of a behavior-specific objective to rank supporting and opposing clients. The paper evaluates agreement with dense references and exact leave-one-client-out retraining, client-removal interventions, audit costs, and information leakage.

Critical strengths
  1. The paper addresses a clear and relevant problem and offers a simple way to obtain behavior-specific client rankings without revealing dense numerical updates to the auditor.
  2. The evaluation is broad, covering dense references, exact leave-one-out retraining, client removal, ablations, multiple models and PEFT settings, system costs, and leakage tests.
  3. The paper clearly states its limitations, including hidden-tail ambiguity, unavailable quantities in its sufficient conditions, and the absence of a formal privacy guarantee.
Critical weaknesses
  1. The methodological novelty over existing federated attribution and auditing methods (e.g., TraceFL (ICSE 2025), ProToken (MLSys 2026), INDEXGUARD (ICML 2026)) is insufficiently established. (Q1)
  2. Audit records may not describe the actual training update. Section 2.4 leaves this binding outside scope. Although the zero-knowledge proof in Appendix B verifies that an audit signature matches a committed hidden update, it does not establish that this committed update is the same update actually submitted to secure aggregation. A malicious client could therefore submit one update for training and a different one for auditing, unless an additional verifiable-input mechanism is integrated. (Q2)
  3. Appendix B tests a small proof circuit, not the millions of LoRA coordinates used in the experiments. Its timings therefore do not establish that clients can afford the proof during training. (Q3)
  4. The practical reliability of identifying the most influential individual client remains limited. In Table 2's ten-client AG News experiment, AUDITRACE ranks the client with the largest leave-one-out effect first for only 39% of queries, despite strong overall ranking correlation. (Q4)
  5. Attribution under client dropout is untested. Appendix D uses full participation. It is unclear how rankings change when clients participate in different subsets of rounds. (Q5-6)
  6. The manuscript needs more proofreading before submission. Several grammatical and formatting errors include but are not limited to:
  • on page 1, "Against exact leave-one-client-out retraining it attains" -> ... retraining, it achieves
  • on page 1, "within 0.020 Spearman of dense cosine", do you mean "within 0.020 in Spearman correlation of the dense-cosine baseline"?
  • on page 1, "A 5% signature retains Spearman 0.860" -> A 5% signature retains a Spearman correlation of 0.860
  • on page 6, what is the "IndexTrace" in fig 3?
  • on page 15, "the final condition in Equation equation 13"
  • on page 15, "with the circuit written in Circom (?)", what does ? mean, maybe a missing citation?
  • on page 23, "increasing the number of monotonically improves", number of what?
Questions for authors
  1. What are the essential methodological differences between AUDITRACE and prior approaches such as TraceFL, ProToken, and INDEXGUARD? Beyond applying signed Top-K overlap to behavior-conditioned gradients, what fundamentally new capability or attribution principle does AUDITRACE introduce? Why are they not compared in evaluation? Could the authors provide more direct comparisons with them under matched information-disclosure budgets?
  2. How can the auditor verify that the update described by the audit record is the same update submitted to secure aggregation?
  3. What are the expected proof-generation time, verification cost, memory consumption, and communication overhead when the circuit scales from 32 private inputs to millions of PEFT parameters?
  4. Why does AUDITRACE achieve only 39% Top-1 recovery against exact LOO effects despite relatively high Spearman correlation and AUROC? Under what conditions can auditors reliably identify the most influential individual client rather than merely obtain an approximate overall ranking? Could the authors report Top-1 confidence or ranking margins, evaluate performance across different client populations, and extend counterfactual validation beyond the ten-client AG News setting?
  5. How should client dropouts be handled? When clients participate in only some training rounds, with different clients participating in different rounds, how does attribution accuracy change?
  6. Under partial client participation, does Eq. (4) rank average alignment per active round or cumulative influence? Could the authors test how unequal participation affects the resulting rankings?
5 comments
AuthorsOpenReview ↗

Response to Reviewer nYzp on methodological distinctions and additional verification evidence

e sincerely thank you, Reviewer nYzp, for recognizing the value of behavior-specific client attribution, the simplicity of the method, and the breadth of its validation. Your suggestions helped us clarify what AUDITRACE adds to prior work and strengthen the evaluation with a sign ablation, proof benchmarks up to 65,536 coordinates, and Top‑2 and Top‑3 shortlist results. (colored in the revision).

W1/Q1: Novelty over TraceFL, ProToken and IndexGuard

AUDITRACE is, to our knowledge, the first method combining three properties: (i) the behavior is specified post hoc as any differentiable objective; (ii) the output is a signed ranking separating supporting from opposing clients; (iii) no individual dense update is observed, reconstructed or estimated (Sec. 2.3).

TraceFL / ProTokenIndexGuardAUDITRACE
QuestionWho caused this prediction / token?Is this client a backdoor attacker?Who supported or opposed a behavior chosen after training?
Needs individual client modelsYesNoNo
Query-conditioned, signedNoNoYes

TraceFL and ProToken trace neuron provenance through individual client models, which secure aggregation withholds, so they cannot run under a matched disclosure budget. IndexGuard is a training-time, query-free screening defense. It produces no behavior-conditioned ranking.

The new principle is that signs of the sparse support, matched to a query gradient, carry the attribution signal. We clarified these differences in method and access to client information in Section 2.3 (Related Work), particularly in the Positioning paragraph.

W2/Q2: Is the audited update the one submitted to secure aggregation?

Appx. B now specifies this binding. The client commits once to its update; an adapted EIFFeL input validation accepts an aggregation input only if it opens that commitment, and the Groth16 proof shows the record is the signed Top-K of the vector opening the same commitment. Commitment binding then makes a "train one update, audit another" attack fail. In our implementation tests, 13 of 13 tampered-record cases are rejected and none of 2,310 fabricated candidates is accepted (Table 7).

W3/Q3: Proof cost at scale

We replaced the 32-input prototype with a dimension-parameterized circuit, measured across four vector sizes spanning a 64x range up to a full LoRA projection (2,000x larger than before), (Appx. B, Table 8). Verification is constant: about one second and 1 GiB at every size. Communication is constant and negligible: the proof and public inputs total about 1 KB per client and audited round, about 0.1% of the audit record they certify. Proving time grows sublinearly (22x for a 64x larger vector) and is paid off the training path.

W4/Q4: Top-1 recovery of 39%

39% does not mean the top contributor goes undetected. We added Top-k recovery and margins (Table 3):

vs. exact LOOUses dense valuesSpearmanTop-1Top-2Top-3
AUDITRACENo0.8270.3900.7700.833
Dense cosineYes0.8470.4000.7900.840
Client-FisherYes0.7740.2900.6650.677
Random-0.0020.1100.1800.297

The leading clients are near-ties (median first-second gap: 6.6% of the effect range; AUDITRACE's median top-1 regret: 3.4%), so adjacent clients swap even with full dense updates. With better-separated clients, Top-1 reaches 0.60-0.91 across five PEFT variants (Table 21), and the audit-score gap tells the auditor when one client can be named (AUROC 0.71). Fidelity holds from 50 to 300 clients (Spearman 0.895-0.961; Appx. D.3.2), and the BeaverTails client-removal intervention validates counterfactual effects beyond AG News (Table 14). We also extended exact LOO to 1% and 5% budgets: a 5% record keeps a Spearman correlation of 0.784 (Appx. D.2.2).

W5/Q5-6: Client dropout and unequal participation

New experiments (Appx. E.7):

Participation p1.00.90.80.70.60.7, phase-skewed
Spearman0.9140.8980.9070.8850.8880.888
AUROC0.9920.9910.9890.9850.9800.984

Accuracy stays within 0.03 of full participation. Eq. (4) ranks average alignment per active audited round, not cumulative influence; replacing it with the cumulative sum changes Spearman only from 0.885 to 0.896. Under the phase-skewed schedule the ranking changes by less than 0.01.

W6: Proofreading

Thank you for catching these. All seven items are fixed (abstract wording, the Figure 3 label, "Equation equation", the Circom citation, the missing noun), and we proofread the whole paper for the same patterns (consistent "a Spearman correlation of", headings, captions and references).

We are grateful for this thorough review, which made the paper clearer and stronger.

Thank you for the detailed response. Some concerns still remain:

  1. Appd B addresses the update-binding concern at the protocol level. Could the authors clarify whether Table 7 tests the integrated protocol in the case where a valid audit proof corresponds to an update different from the one submitted for aggregation?

  2. The new circuit benchmarks are useful, but it remains unclear whether they include commitment generation, EIFFeL input validation, and the associated communication. Will the additional commitment binding significantly affect the original AUDITRACE pipeline, e.g., PEFT update size, audit frequency, update encoding?

  3. Do the proof-cost benchmarks in Table 8 include the overhead of the new audit-to-aggregation binding mechanism?

AuthorsOpenReview ↗

Response to Reviewer nYzp: input-binding test and full binding costs

Thank you, Reviewer nYzp, for the careful follow-up and for finding the new circuit benchmarks useful. We revised Appendix B accordingly (changes in red): Table 7 now includes the requested substitution test with EIFFeL validation, and Table 8 has two new panels with the binding costs. No retraining was needed: the added rows of Table 7 and panels B and C of Table 8 are computed on client updates already saved from the paper's existing training runs, using the Appendix B circuit templates; only the proofs and validation checks were run on these saved updates.

1. Does Table 7 test a valid audit proof paired with a different aggregation input?

Yes, the revised Table 7 now includes this case. The client logs a commitment to u with an accepting audit proof for u, then submits a different encoded vector v ≠ u as its aggregation input; EIFFeL validates the actual secure-aggregation input against the logged commitment.

Outcome (20 client–round attempts)Result
Audit proofs for u that verify20/20
Substituted inputs v ≠ u admitted0/20
Honest control inputs admitted20/20

Setup: n = 1,024, K = 103, real layer-0 q-projection LoRA-A updates from the AG News/Llama-3.2-1B run, 10 clients at rounds 5 and 10, signed 32-bit encoding with one scale per round. In every attempt the proof for u verifies, yet input validation rejects v, while all honest inputs are admitted.

2.1 Do the benchmarks include commitment generation, EIFFeL input validation and communication?

Yes. The new Table 8C reports both Groth16 relations for all 128 LoRA factors of one real client update (M = 3,407,872 coordinates; 16 cores, factors processed sequentially):

Full adapterCommitmentWitness + provingVerificationProof + public inputs
Audit relation144.2 s131.85 min147.7 s135,871 B
Input opening, Eq. (14)142.7 s70.21 min126.8 s124,161 B

All 128 proofs verify for each relation. These are single-node times with the factors proved one after another. The 128 factor proofs are independent, so running them in parallel across nodes reduces the wall-clock time roughly in proportion to the number of nodes; the total compute is unchanged.

  • Commitment generation. Inside the audit proof, the commitment opening is 380,829 of 5,120,753 constraints (7.44%) at n = 65,536. Native computation takes 2.4 s per 65,536 coordinates (Table 8B).
  • EIFFeL input validation. The input-opening proof for Eq. (14) has 2,477,981 constraints at n = 65,536; proving takes 71.2 s and verification 0.95 s (Table 8B), and 70.21 min / 126.8 s for the full adapter.
  • Communication. Each proof is about 805 B plus about 0.26 KB of public inputs; the ~1 KB in our first reply is this per-proof size. A full adapter uses one proof per factor, so 136 KB for audit proofs and 124 KB for input-opening proofs, with the commitment included as a public input.

2.2 Does the binding significantly affect the pipeline (PEFT update size, audit frequency, update encoding)?

No. All three are unchanged:

  • PEFT update size. The adapter keeps its M = 3,407,872 trainable coordinates; the binding adds only the proof and public-input files listed in 2.1.
  • Audit frequency. The audited rounds are unchanged. Audit proofs can be generated after the round; input validation precedes acceptance of the aggregation input.
  • Update encoding. The signed 32-bit encoding keeps every selected index and sign in all 120 tested records (10 clients × 2 rounds × 3 budgets × 2 selection rules), so the replayed attribution scores are unchanged.

Because proofs are generated per LoRA factor, certified records use per-factor Top-K; at a 10% budget, Spearman correlation with the dense reference is 0.787 [0.751, 0.822], versus 0.808 [0.776, 0.840] for global Top-K.

3. Does Table 8 include the overhead of the binding mechanism?

Yes, the revised Table 8 now includes it. The original Table 8 measured the audit relation, which already contains the commitment opening (7.44% of constraints at n = 65,536); reusing that commitment for binding adds no constraints to it. New panels B and C add the remaining client-side binding costs: panel B gives native commitment computation and the Eq. (14) input-opening relation per 65,536 coordinates, and panel C gives both relations for the full adapter. We also renamed the "Complete" column to "Audit total" to make its scope explicit.

We hope the direct substitution test and the full cost breakdown answer the remaining questions. Your follow-up made Appendix B considerably more complete, and we would be grateful if you would reconsider your assessment in light of these additions.

Reviewer_MctmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies post-hoc client attribution in federated parameter-efficient fine-tuning. It identifies which clients’ updates support or oppose a specified model behavior without exposing their dense numerical updates to the auditor. The authors propose AUDITRACE, which separates the learning and audit information paths. At selected checkpoints, each client records the indices and signs of its Top-K update coordinates by absolute magnitude. The auditor defines a differentiable objective for the behavior of interest, constructs the corresponding gradient signature, and calculates normalized signed-overlap scores. These scores incorporate FedAvg weights and are aggregated across checkpoints and queries to produce a directional client ranking.

Critical strengths
  1. AUDITRACE ranks clients by whether their updates support or oppose a specified objective using only Top-K coordinate indices and signs, without exposing update magnitudes to the auditor. Client records can be reused for different subsequent queries, offering a useful trade-off between dense-update confidentiality and post-hoc audibility.

  2. Beyond comparisons with a dense first-order reference, the paper also evaluates exact one-client-out experiment and client-removal experiment. Its ranking correlation with exact LOO effects reaches 0.827, close to the dense cosine baseline 0.847. These results provide empirical evidence that sparse signed records preserve useful information of clients' influence.

Critical weaknesses
  1. The method discards updated magnitudes and unselected coordinates, although this information can change the direction and ranking of actual influence. Its theoretical guarantees rely on conditions such as small tail contributions and similar magnitudes of retained coordinate products, which the auditor cannot verify. LOO validation is largely restricted to a single AG News configuration, and Top-1 recovery of the client with the largest positive effect is only 39%. It therefore remains unclear when the method can reliably identify specific clients.

  2. The zero-knowledge proof in Appendix B only establishes consistency between the signature and the committed hidden update. it, however, does not ensure that this update is the actual input to secure aggregation. Even with a fully truthful signature, Appendix C.10 shows that unretained coordinates can cause substantial influence that the signature fails to capture. The paper lacks experiments evaluating such strategic evasion, leaving unclear the extent to which truthful signatures can support trustworthy auditing.

Questions for authors
  1. The authors should use the available dense updates to report how often the theoretical sufficient conditions hold, and analyze whether tail contributions, magnitude dispersion, and cross-client scale differences explain attribution failures?

  2. In the existing LOO configurations, what ranking correlation, sign accuracy, and Top-1 recovery rate do the 1%, 5%, and 10% signatures achieve? The current budget experiments primarily compare against a dense first-order reference, leaving unclear whether performance at lower budgets carries over to counterfactual effects.

  3. Without binding the committed update to the actual secure-aggregation input, what specific guarantees does the current mechanism provide against malicious client behavior? For clients that submit truthful signatures but deliberately shift target-related influence to unretained coordinates, the authors must characterize their ability to evade auditing?

1 comment
AuthorsOpenReview ↗

Response to Reviewer Mctm on complementary results across audit budgets

Thank you, Reviewer Mctm, for recognizing the value of reusable audit records that keep dense updates private, and the strong agreement with exact LOO effects. Your suggestions prompted complementary analyses of exact LOO agreement at 1%, 5%, and 10% audit budgets and when the theoretical bounds certify attribution. (colored in the revision).

W1/Q1: Discarded magnitudes and tail: do they explain failures? How often do the sufficient conditions hold?

We ran exactly the requested analysis on dense instrumented updates (Sec. 5.2, Table 2; Appx. C.12). We restore the discarded information one error term of the bound at a time:

Score (held-out, K/M = 10%)Sign errorPair errorSpearman
AUDITRACE (IDs + signs)0.1360.1720.808
+ each client's scale0.1360.1590.830
+ exact retained magnitudes0.1490.1650.818

Discarding magnitudes costs almost nothing: exact magnitudes do not reduce sign errors, and client scale adds only +0.022. We then tested which factor predicts the remaining errors (Table 11):

DiagnosticError rate, worst 20%Error rate, best 20%AUROC for errors
Magnitude dispersion0.1500.1870.47 (chance)
Cross-client scale0.1800.1530.54 (chance)
Effect cancellation0.3670.0200.80
Overlap score (auditor-observable)0.3800.0130.80

Dispersion and scale do not explain failures. Errors sit on effects whose contributions nearly cancel; weighted by effect size, sign accuracy is 93.8%. The auditor can identify reliable attributions without dense updates: the observable overlap score separates error rates of 38.0% and 1.3%. On dense updates, the record matches the dense effect sign in 86.4% of held-out cases (Appx. C.12).

W1: Top-1 recovery of 39% and LOO scope

A Top-1 of 39% reflects near-ties: the median first-second effect gap is 6.6% of the effect range, and dense cosine with full updates reaches only 40%. AUDITRACE recovers 77.0% of the true top-2 and 83.3% of the true top-3 (dense cosine 79.0% / 84.0%; Table 3). So 39% does not mean the top contributor goes undetected; only adjacent, nearly identical clients swap, consistent with Spearman 0.827. With better-separated clients, Top-1 reaches 0.60-0.91 across five PEFT variants (Table 21). Beyond AG News, the client-removal intervention on BeaverTails validates counterfactual effects: removing AUDITRACE's top-5 clients improves the unsafe-target objective by 0.0232 vs. 0.0230 for the dense oracle and 0.0129 for random (Table 14).

Q2: LOO agreement at 1%, 5%, 10% budgets

We ran this experiment in the same exact-LOO configuration as Table 3 (AG News/Llama-3.2-1B, ten clients, five seeds), scoring the same retrained models at each budget (Appx. D.2.2, Table 15):

vs. exact LOOSpearmanSign acc.AUROCTop-1Top-3
AUDITRACE, 1%0.5310.7020.8120.2500.640
AUDITRACE, 5%0.7840.8060.9210.3500.790
AUDITRACE, 10%0.8270.8340.9410.3900.833
Dense cosine (full updates)0.8470.8710.9520.4000.840

Performance at lower budgets carries over to counterfactual effects: a 5% record keeps 95% of the 10% correlation at half the payload, and the LOO curve has the same shape as the dense-reference curve (Table 5).

W2/Q3: Guarantees without input binding, and evasion with truthful signatures

  • What the proof guarantees. An accepted record is exactly the signed Top-K of the committed update: all 13 tampered-record cases are rejected (flipped signs, swapped or duplicate indices, altered records) and none of 2,310 fabricated candidates is accepted (Appx. B, Table 7). Across a 64x range of vector sizes, verification stays at about one second and the proof adds only about 0.1% to the audit record (Table 8).
  • Binding to the SecAgg input. Appx. B now specifies the composition with EIFFeL's input validation: aggregation accepts an input only if it opens the same commitment that the audit proof uses, so commitment binding forces the trained and audited updates to be identical.
  • Strategic evasion with truthful records: new experiment (Appx. F.1-F.2). We built a white-box adversary that keeps its record unchanged and optimizes hidden coordinates under a norm budget (40 cases, 30 clients). The achievable gain is small (median 3.8-4.7 x 1e-4 in log-likelihood, with 95% intervals that include zero) and costs the adversary utility. A randomized-record profile further hardens the audit at equal record size:
Record (equal bytes)Evasion cases recoveredCases lostMedian audit rank of attacker
Deterministic Top-K--14.5
75% head + 25% sampledabout 20%08 / 7

We thank the reviewer for these precise questions, which made the paper clearer.

Reviewer_TLq3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper considers an accountability problem in federated PEFT: when a developed FL model exhibits unexpected behaviors, an auditor should determine which client promoted or opposed that behavior without disclosing individual updates. Authors proposed an AUDITTRACE, which computes the gradient of a differentiable objective representing the misbehavior, extracts its Top-KK signed mask, and ranks clients by computing the FedAvg-weighted signed overlap across checkpoints. Authors also provided theoretical approximation bound and validated the method empirically against other methods.

Critical strengths
  • The idea of stripping away continuous magnitudes and keeping only Top-KK indices and signs is a simple yet effective mechanism, with strong signal shown in Table 1 and 13.
  • The paper's evaluation settings (except for models) are comprehensive, covering different PEFT variants (LoRA, DoRA, AdaLoRA, rsLoRA, LoRA+), data heterogeneity levels, and explicit leakage evaluations.
Critical weaknesses
  • In Section 3.3 and Appendix B, the authors introduce a Groth16 zk-SNARK circuit to prevent malicious clients from fabricating benign Top-KK signatures. However, the prototype is only done on a toy example with M=32M=32, whereas Llama-3B LoRA updates have M>9×106M > 9 \times 10^6 coordinates. As acknowledged in Appendix B, this proof does not bind the committed vector to the actual input entering SecAgg. Thus, against an active adversary who either falsifies the audit record or hides malicious mass in the tail (Proposition 5), the proposed framework wouldn't do well.
  • The empirical evaluation of chosen LLMs is limited. Llama-3.2-1B on AG News is an very outdated benchmark that has trivial implication and significance. Authors should try modern models (at least Qwen3.5 or Gemma 4 models).
  • While overall ranking correlation (ρS=0.827\rho_S = 0.827) and positive-effect AUROC (0.9410.941) against exact LOO are strong, the exact Top-1 client recovery is only 39%39\% (Section 5.3 / D.2.1). While the authors fairly note that the score is meant for overall ranking rather than single-client blame, this limitation should be highlighted and explained in detail for better understanding.
Questions for authors
  • How does AUDITRACE perform when clients train for multiple local epochs (e.g., 5 epochs) or when only a fraction of clients (e.g., 20%20\%) participate per round? Does the normalized temporal aggregation in Eq. (4) remain calibrated when clients participate at different stages of convergence?
  • If a malicious client intentionally caps its malicious coordinate updates just below its Top-KK threshold to evade the audit mask, would randomized coordinate sampling (e.g., importance sampling or randomized sketching instead of deterministic Top-KK) mitigate this vulnerability?
1 comment
AuthorsOpenReview ↗

Response to Reviewer TLq3 on security clarifications and additional tests

Thank you, Reviewer TLq3, for recognizing the technical soundness of AUDITRACE and the strong attribution results from a simple signed Top‑K record. Your suggestions helped us clarify the security guarantees and add complementary evidence from Qwen3.5 and Gemma 4, multiple local epochs, and partial participation.

W1: Binding to the SecAgg input, proof scale, and active adversaries

  • Binding. Appx. B now specifies a composition with EIFFeL's secure-aggregation input validation: the client commits once to its update; aggregation accepts an input only if it opens that commitment, and the Groth16 proof shows the audit record is the signed Top-K of the vector opening the same commitment. Commitment binding then forces both channels to use the same update. In our implementation tests, all 13 tampered-record cases are rejected and none of 2,310 fabricated candidates is accepted (Table 7).
  • Scale. We replaced the prototype with a dimension-parameterized circuit and measured it across four vector sizes spanning a 64x range, up to a full LoRA projection (2,000x larger than the original prototype; Appx. B, Table 8). Verification time stays constant at about one second, the proof stays constant in size and adds only about 0.1% to the audit record it certifies, and proving time grows sublinearly (22x for a 64x larger vector), off the training path.
  • Hidden-tail adversary. We added a white-box stress test in which a client keeps a truthful record while optimizing its hidden tail (Appx. F.1): the achievable gain is small and costs the adversary utility, and a randomized-record profile hardens the audit further (see Q2).

W2: Modern backbones (Qwen3.5 / Gemma 4)

We ran both requested families. Fidelity on the newest, larger models matches the original results (Table 4; setup in Appx. D.1.2):

BeaverTails, K/M = 10%SpearmanSign acc.AUROCNDCG@5
Llama-3.2-3B (original)0.8860.8770.9690.946
Qwen3.5-9B (new)0.8810.8720.9580.901
Gemma-4-12B (new)0.8870.8790.9620.907

W3: Top-1 recovery of 39%

39% does not mean the top contributor goes undetected. We now report Top-k recovery (Sec. 5.2-5.3, Table 3):

vs. exact LOOUses dense valuesTop-1Top-2Top-3
AUDITRACENo0.3900.7700.833
Dense cosineYes0.4000.7900.840
Random-0.1100.1800.297

The leading clients are near-ties (median first-second gap: 6.6% of the effect range), so adjacent clients swap even with full dense updates, consistent with the high Spearman correlation (0.827). Where leading clients are better separated, Top-1 is high: 0.60-0.91 across the five PEFT variants on RoBERTa (Table 21). Errors concentrate on near-zero effects and can be flagged by the auditor (AUROC 0.80; Sec. 5.2).

Q1: Multiple local epochs, partial participation, and Eq. (4)

We added both studies (Appx. E.6-E.7). Fidelity remains high in all settings:

SettingSpearmanAUROCNDCG@5
E = 1 / 3 / 5 local epochs0.914 / 0.899 / 0.8850.993 / 0.968 / 0.9580.943 / 0.941 / 0.945
p = 1.0 / 0.8 / 0.6 participation0.914 / 0.907 / 0.8880.992 / 0.989 / 0.9800.945 / 0.936 / 0.905
p = 0.7, phase-skewed schedule0.888 (uniform: 0.885)0.9840.908

In the phase-skewed schedule, half of the clients train mostly in early rounds and the other half mostly in late rounds; the ranking changes by less than 0.01, so Eq. (4) remains comparable across clients audited at different stages of convergence. The gap to the dense reference stays constant, so the sparse signature adds no extra error.

Q2: Would randomized coordinate sampling mitigate sub-threshold evasion?

Yes, partially, and we now quantify it (Appx. F.2). At equal record size, a record with 75% Top-K head and 25% uniformly sampled tail coordinates recovers about 20% of the evasion cases that deterministic Top-K misses and loses none, moving the attacker's median audit rank from 14.5 to 8. We present it as an optional hardening profile.

We hope these additions resolve the reviewer's concerns. With the modern backbones, the scaled proof, and the new epoch and participation studies, we believe the paper is now substantially stronger, and we are grateful for the suggestions that led to it.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Gentle reminder about our rebuttal responses

Dear Reviewers TLq3, Mctm, and nYzp,

We posted our responses on November 11, early in the discussion period, so that you would have enough time to read them and, if needed, ask for further clarifications or revisions. We have not received any replies yet, so we are sending a short reminder.

All three reviews recognized the core idea and found no fundamental problems with the technical soundness of AUDITRACE. The remaining concerns were mainly about scope and verification, and we addressed each of them with new experiments and analyses in the revision (changes are colored):

  • Modern backbones: Qwen3.5-9B and Gemma-4-12B (Table 4, Appx. D.1.2)
  • Security: binding the audit record to the secure-aggregation input, and a proof circuit scaled up to a full LoRA projection (Appx. B, Tables 7–8)
  • Attribution reliability: Top-k recovery, and exact leave-one-out at 1%, 5%, and 10% budgets (Table 3, Appx. D.2.2)
  • Training settings: partial participation and multiple local epochs (Appx. E.6–E.7)
  • Strategic evasion: a white-box hidden-tail adversary and a randomized-record hardening profile (Appx. F.1–F.2)

Since these points bear directly on your assessments, we would be grateful if you could look at our responses and, if they resolve your concerns, consider updating your scores. We are happy to answer any further questions before the discussion period ends.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.