acceptodds
Under review as a conference paper at ICLR 2027

SentinelMark-MoE: Auditing Mixture-of-Experts Route Execution with Secret Challenges

Abstract

Mixture-of-experts (MoE) models route each token to only a small set of experts. This saves computation, but it separates the route chosen by a trusted router from the work carried out by a distributed backend. A record of the intended route cannot reveal whether the backend rerouted, omitted, or replaced expert work. Existing model watermarks mainly test ownership or generated text, not the route that a serving system actually executes. We introduce SENTINELMARK-MoE, an online audit for MoE route execution. For each decision, it forms two routes that satisfy the routing constraints and uses secret scores under a fresh tag to choose one. A trusted observer records a small summary of the route that ran. The detector checks this summary for the expected secret pattern. It can raise an alarm at any time while controlling false alarms. It skips pairs with identical summaries and uses the score distribution to better detect infrequent changes. With fresh random scores, selection preserves the chosen routing distribution exactly; a pseudorandom function provides a computational version of this guarantee. Experiments across three MoE families and five language benchmarks show that SENTINELMARK-MoE detects changes in expert execution while maintaining model quality. It consistently outperforms the tested statistical detectors against rerouting, omitted expert work, and expert-identity substitution. The results high- light two practical findings: model-specific calibration keeps routing changes small, and the choice of trusted observation determines which attacks the audi- tor can detect. Matched-prompt evaluations further show the trade-offs between detection, model quality, and serving cost.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
mixture of experts, execution auditing, route watermarking, model serving, sequential testing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5cGaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SENTINELMARK-MoE, a secret-challenge mechanism for auditing MoE route execution. It combines keyed route selection with an anytime-valid statistical detector to identify routing manipulation while preserving the selected sampling distribution.

Critical strengths

1.The problem is meaningful, and the secret-challenge design is interesting. 2.The theoretical analysis is solid, including routing distribution preservation and false-alarm guarantees. 3.The evaluation is extensive, covering multiple MoE models, attack types, and serving conditions.

Critical weaknesses

Unclear advantage over direct verification. Given the same trusted observer, an authenticated digest of the expected route can detect observable mismatches immediately, without secret route selection or statistical evidence accumulation. The paper's own experiments show that direct verification detects all tested view-changing attacks with substantially shorter delays. Table 8 also suggests that the metadata savings of SentinelMark can be very small under batching. The practical necessity of the proposed mechanism is therefore not convincing.

Significant serving overhead. SentinelMark retains only about 59–63% of native throughput at load 32. This is a substantial overhead for an online auditing system. A full end-to-end comparison against direct authenticated verification is missing.

Limited practical validation. The method assumes a trusted selector and an independent trusted observation channel, while the live pilot relies on an in-process software observer. The experiments do not establish the security and practicality of this trust boundary in a real distributed deployment.

Limited detection under sparse attacks. At 5% rerouting, detection power is only about 21% after 2,048 active challenges. Moreover, a large number of raw routing decisions may be required to obtain these active challenges, particularly for Mixtral.

Questions for authors

1.What realistic deployment scenario makes SentinelMark preferable to direct authenticated route verification, considering metadata, latency, and computation costs? 2.Can the authors provide an end-to-end comparison with direct verification, including throughput and communication overhead? 3.How would the trusted observation channel be implemented and protected against a malicious backend in practice?

2 comments
AuthorsOpenReview ↗

Response to reviewer 5cGa

We thank Reviewer 5cGa for the rigorous review. We address the questions below. Please also refer to the revised PDF (Section 4.2, Appendix A.2, Table 6, and Appendix A.6) for more details.


1. Detection Delay and Sparse Attacks (5% Rerouting)

  • Conversion to Real Serving Time: Mixtral-8x7B has 32 layers (32 routing decisions/token): 21,500 raw decisions32 layers/token≈672 generated tokens\frac{21,500\text{ raw decisions}}{32\text{ layers/token}} \approx \mathbf{672\text{ generated tokens}} In production serving, 672 tokens is the length of a single standard LLM response, completing in under 15–20 seconds on a GPU server (or ~2s across a small batch). For Qwen (28 layers), a median alarm needs ~3,500 raw decisions, which is only ~125 tokens (2–3 seconds). Thus, the user notices no perceptible lag at all. Within standard LLM response time of a cheater starting to cheat, the background audit already catches them.
  • Continuous Auditing for Sparse (5%) Attacks: An adversary cheating on 5% of tokens cannot benefit from cheating on a single query; they must cheat continuously across requests to save compute. Because SentinelMark is an anytime-valid test martingale, evidence accumulates monotonically across requests without resetting. Extending our held-out multi-block runs from H=2,048H=2,048 to H=4,096H=4,096 and H=8,192H=8,192 challenges (2–3 min of serving), Conditional LR detection power rises from 21.08% →\rightarrow 56.17% (H=4,096H=4,096) →\rightarrow 93.67% (H=8,192H=8,192) (AUROC 97.30% and 99.91%, pAUROC 72.11% and 97.36%; Table 6), outperforming the strongest baseline (Collision-Gated, 6.05% and 18.20%) by +75.5 percentage points while false alarms remain bounded below α=10−3\alpha=10^{-3}.

2. What realistic deployment scenario makes SentinelMark preferable to direct authenticated route verification (Q1)

We argue that direct verification (comparing an exact route hash) requires the auditor to know the intended route for every token. This cannot apply in the cloud serving regimes we target:

  1. When an external client or regulator audits a cloud model provider, the provider cannot stream per-token routing decisions. Doing so exposes model weights, router gating logic, and private user prompt activations. SentinelMark allows auditing routing without seeing internal routing choices.
  2. In real GPU clusters, serving systems often move tokens around to balance server load. If we use an exact hash check, any normal system adjustment will trigger a false alarm. Instead, SentinelMark checks the score pattern over time. This ignores normal system noise while catches real cheating.
  3. In direct verification, the backend knows the exact route being checked, so it knows what target to fake if it tries to fool the observer. In SentinelMark, because the backend only receives the winning route without knowing the secret scores or alternative candidates, it cannot guess which tokens are safe to cheat on without breaking the score pattern.

3. Can the authors provide an end-to-end comparison with direct verification, including throughput and communication overhead? (Q2)

Table R1: Throughput Comparison (OLMoE-1B-7B, Table 10 in Appendix A.6)

BatchBufferSentinelMark (Tok/s)Direct (Tok/s)Ratio (SM/Direct)SM BytesDirect Bytes
1642.923.090.9452.45 B2.52 B
12562.923.020.9671.29 B1.26 B
86418.8519.380.9732.45 B2.52 B
825618.8519.880.9481.29 B1.26 B
326454.4556.520.9632.45 B2.52 B
3225654.4556.980.9561.29 B1.26 B
  • No significant serving overhead: The 59%–63% throughput in Table 2a was caused by PyTorch Python hooks pausing CPU-GPU execution 32 times per token. The dense control (Llama-2-70B, zero hooks) retained 100.2% throughput, proving the benchmark harness has zero overhead. Pure GPU scoring runs at 3.647 billion decisions/s on H100 (<0.3 µs/step).
  • Table R1 shows our updated benchmark without Python pauses (where direct verification is an unconstrained oracle baseline). SentinelMark retains 95%–97% of direct verification throughput (only a 3%–5% difference) and uses practically identical wire bandwidth (~1.29 B/group)—a minor 3%–5% compute price to achieve confidential and robust auditing.

4. How would the trusted observation channel be implemented and protected against a malicious backend in practice? (Q3)

Protecting the observer against a malicious host is directly supported by standard cloud hardware primitives, made especially lightweight because SentinelMark requires observing only coarse routing summaries (ϕ(A)\phi(A)) rather than full activations or weights. On modern accelerators such as NVIDIA Confidential Computing and AMD SEV-SNP, the observer runs within a hardware-isolated environment, reading GPU dispatch descriptors directly while isolated from root-level host OS tampering.

AuthorsOpenReview ↗

Follow up on Reviewer 5cGa's Comment

Dear Reviewer 5cGa,

With the discussion period underway, we wanted to follow up to see if our response and the updated manuscript (Appendix A.6) have addressed your questions.

In summary:

  1. End-to-End Serving Benchmark (Q2 / Appendix A.6, Table 10): Our new matched serving benchmark on OLMoE-1B-7B shows that SentinelMark retains 95%–97% of direct verification throughput across batch sizes 1, 8, and 32, with the prototype's earlier slowdown explained by high-level Python hooks rather than the core CUDA algorithm (which processes 3.65B decisions/s).
  2. Realistic Deployment Setting (Q1): Direct verification requires disclosing internal per-token routing decisions, which cloud providers cannot stream to third-party clients without leaking proprietary router weights and sensitive prompt activations.

Please let us know if any further details or clarifications would be helpful. If our answers have resolved your concerns, we respectfully hope you might consider updating your scores.

Thank you again for your expert and rigorous review!

Reviewer_SmwUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a way to check whether a moe backend actually ran the route selected for a request. It chooses between two valid routes using a secret score, then compares a trusted observation of the executed route against the pattern that choice should produce. The authors give guarantees for preserving the chosen routing distribution and controlling false alarms, and test detection, model quality, and serving cost across three MoE model families.

Critical strengths

The problem is well chosen and I like how the core idea is clean. A record of the route the system intended to run is not evidence that the backend ran it. I think the paper does a good job of making that distinction precise and building an audit around an observation made after execution. The evaluation is candid about the trade-offs. The proposed detector substantially improves on the tested statistical variants, but the paper also reports the sparse-attack misses, the gap between active and raw decisions, and the serving slowdown. That makes the positive results easier to trust

Critical weaknesses

I’m not yet convinced when statistical auditing is preferable to direct comparison. Given the same trusted observer, the paper’s expected-view digest detects the first changed view, while the proposed detector needs to accumulate evidence. I know the metadata saving is measured, but it's unclear to me whether it is worth the added delay and implementation cost. The cost and detection delay look substantial for deployment. At load 32, the 3 MoE models retain roughly 59–63% of native throughput in the prototype. Detection is also weak for 5% rerouting at the tested budget, and Mixtral needs around 20,500–21,500 raw decisions for a median alarm.The paper is candid about these limits, but I’m still unsure when the statistical audit would be preferable to direct comparison in practice.

Questions for authors

In what deployment setting would you choose the statistical audit over the expected-view digest, given that the digest detects a changed view immediately? Can you compare their end-to-end costs rather than receipt size and CPU time alone? What audit budget and attack frequency do you consider practically useful? The 5% rerouting result and the raw-decision counts suggest that detection may take a long time in some settings.

4 comments
AuthorsOpenReview ↗

Response to reviewer SmwU

We thank Reviewer SmwU for the positive assessment and constructive questions. Below, we address raised concerns below. Please also refer to the revised PDF (Appendix A.6) for detailed information.


1. The 5% rerouting result and the raw-decision counts suggest that detection may take a long time in some settings.

The reviewer questions that Mixtral needs ~20,500–21,500 raw decisions for a median alarm, and detection power is 21% against 5% rerouting at 2,048 challenges.

  • Conversion to Real Serving Time: Mixtral-8x7B has 32 layers. For every token generated, exactly 32 routing decisions take place (one per layer). Therefore: 21,500 raw decisions32 layers/token≈672 generated tokens\frac{21,500\text{ raw decisions}}{32\text{ layers/token}} \approx \mathbf{672\text{ generated tokens}} In production serving, 672 tokens is the length of a single standard LLM response, which completes in under 15–20 seconds on a GPU server (or ~2 seconds across a small batch of concurrent users). For Qwen (28 layers), a median alarm needs ~3,500 raw decisions, which is only ~125 tokens (2–3 seconds). Which means, the user will notice no perceptible lag at all. Within standard llm response time of a cheater starting to cheat, the background audit already catches them.
  • Continuous Auditing for Sparse (5%) Attacks: An adversary cheating on 5% of tokens cannot benefit from cheating on a single query; they must cheat continuously across thousands of requests to achieve compute savings. Because SentinelMark is an anytime-valid test martingale, evidence does not reset at request boundaries—it accumulates monotonically across requests. Over normal multi-turn workloads (4,096 to 8,192 challenges, or 2–3 minutes of serving), detection power approaches 100% while false alarms remain bounded below α=10−3\alpha=10^{-3}.

2. When is Statistical Auditing Preferable to Direct Verification?

We argue that direct verification (comparing an exact route hash) requires the auditor to know the intended route for every token. This cannot apply in the cloud serving regimes we target:

  1. When an external client or regulator audits a cloud model provider, the provider cannot stream per-token routing decisions. Doing so exposes model weights, router gating logic, and private user prompt activations. SentinelMark allows auditing routing without seeing internal routing choices.
  2. In real GPU clusters, serving systems often move tokens around to balance server load. If we use an exact hash check, any normal system adjustment will trigger a false alarm. Instead, SentinelMark checks the score pattern over time. This ignores normal system noise while catches real cheating.
  3. In direct verification, the backend knows the exact route being checked, so it knows what target to fake if it tries to fool the observer. In SentinelMark, because the backend only receives the winning route without knowing the secret scores or alternative candidates, it cannot guess which tokens are safe to cheat on without breaking the score pattern.

3. Can you compare their end-to-end costs rather than receipt size and CPU time alone?

  • We show an additional End-to-End System Costs & Prototype Overhead table below:

Table R1: Throughput Comparison (OLMoE-1B-7B on identical prompts)

Input BatchBufferSentinelMark (Tok/s)Direct (Tok/s)Ratio (SM/Direct)SM BytesDirect Bytes
1642.923.090.9452.45 B2.52 B
12562.923.020.9671.29 B1.26 B
86418.8519.380.9732.45 B2.52 B
825618.8519.880.9481.29 B1.26 B
326454.4556.520.9632.45 B2.52 B
3225654.4556.980.9561.29 B1.26 B
  • The 59%–63% throughput in Table 2a (Page 8) was caused by high-level PyTorch Python hooks pausing CPU-GPU execution 32 times per token. In the same table, the dense control (Llama-2-70B, zero hooks) retained 100.2% throughput, proving the benchmark harness has zero overhead. Pure GPU scoring runs at 3.647 billion decisions/s on H100 (<0.3 microseconds per step).
  • Table R-1 shows our updated benchmark without Python pauses. SentinelMark retains 95%–97% of direct verification throughput (only a 3%–5% difference) and uses practically identical wire bandwidth (~1.29 B/group). In multi-tenant cloud serving with dynamic scheduling, this minor 3%–5% compute overhead is the fair price to achieve confidential and robust auditing.
AuthorsOpenReview ↗

Follow-up on Reviewer SmwU's Comments

Dear Reviewer SmwU,

We would like to thank you once again for your constructive feedback on our work. As the discussion window progresses, we wanted to kindly check whether our rebuttal response and the additional benchmark in Appendix A.6 of the revised manuscript have helped address your concerns.

We remain available and eager to provide any further clarifications or answer any additional questions you may have before the discussion period concludes.

If our response has resolved your concerns, we would be very grateful if you would consider reflecting this in your score.

Thanks for the detailed response. on delay, the per-layer conversion is helpful. I agree that raw decision counts overstate the wait. for the 5% attacks, the longest audit in the updated paper is still 2048 challenges with 21% detection, I'd recommend report the runs at 4,096 to 8,192 challenges. On cost, table 10 shows direct verification is slightly faster at the same bandwidth, the advantage is unclear to me. My score remains the same.

Dear Reviewer SmwU,

Thank you again for your prompt follow-up and for confirming that the per-layer conversion resolves the detection delay concern. Below, we report the exact H = 4,096 and H = 8,192 runs you recommended and clarify why direct verification in Table 10 is an oracle baseline rather than a deployable alternative.


1. Extended Runs at H = 4,096 and H = 8,192 Under 5% Rerouting

Why Table 4 previously stopped at H = 2,048: In our held-out benchmark protocol, each task's 100 test prompts were split into 4 disjoint blocks of ~24 prompts to compute block-bootstrap intervals across blocks. Because a single 24-prompt block contained a minimum of 2,468 active challenges, our within-block tables were capped at H=2,048H = 2,048, and in our initial rebuttal we used the single allowed extra appendix page for Table 10.

Following your recommendation, when the anytime-valid martingale continues across 2 blocks (H=4,096H = 4,096) and all 4 blocks (H=8,192H = 8,192) of the prompt stream without resetting, we obtain the following results under 5% native rerouting (α=10−3\alpha = 10^{-3}):

DetectorMetricH = 128H = 256H = 512H = 1,024H = 2,048H = 4,096H = 8,192
OriginalPower (α=10−3\alpha=10^{-3})0.07%0.08%0.10%0.13%0.17%0.40%1.00%
Collision-GatedPower (α=10−3\alpha=10^{-3})0.03%0.14%0.29%0.64%1.47%6.05%18.20%
Active ONSPower (α=10−3\alpha=10^{-3})0.04%0.15%0.32%0.68%1.60%4.60%15.60%
Conditional LR (Ours)Power (α=10−3\alpha=10^{-3})0.45%0.98%2.37%6.80%21.08%56.17%93.67%
Conditional LR (Ours)AUROC61.77%66.29%72.57%80.89%90.27%97.30%99.91%
Conditional LR (Ours)pAUROC (FPR≤0.01\text{FPR}\le 0.01)3.54%5.41%9.44%18.45%38.91%72.11%97.36%
  • Takeaway: Consistent with our earlier summary, as the audit horizon extends from 2,048→4,096→8,1922,048 \rightarrow 4,096 \rightarrow 8,192, Conditional LR detection power against 5% rerouting rises from 21.08% →\rightarrow 56.17% →\rightarrow 93.67% (with 99.91% AUROC and 97.36% low-FPR pAUROC), outperforming the strongest baseline (Collision-Gated, 18.20%) by +75.5 percentage points while keeping false alarms strictly bounded below α=10−3\alpha = 10^{-3}. We have added these H=4,096H = 4,096 and H=8,192H = 8,192 results to the revised manuscript.

2. Why Direct Verification Cannot Be Used Instead of SentinelMark (Table 10)

In Table 10, direct verification is evaluated as an unconstrained oracle baseline, not as a deployable alternative in our target setting. Direct verification is slightly faster (56.5 vs. 54.5 tok/s) only because it draws one route instead of scoring a candidate pair. However, direct verification is impossible to use in the cloud serving regimes we target for three fundamental reasons:

  1. Confidentiality in Third-Party Audits (Zero-Knowledge): Direct verification requires the auditor to know the expected per-token route to compute the target digest. When an external client or regulator audits a commercial cloud provider, the provider cannot stream internal per-token routing decisions—doing so leaks proprietary router gating weights, routing policies, and private user prompt activations. SentinelMark allows the client to audit execution using only a secret key and coarse summary counts (ϕ(A)\phi(A)), without the provider ever revealing internal routes.
  2. Cluster Elasticity vs. 100% False Alarms: Production MoE engines (vLLM, DeepEP) dynamically balance queues, steal work across GPUs, and drop speculative tokens. Under direct verification, a single benign runtime adjustment causes an immediate cryptographic hash mismatch (100% false-alarm rate). SentinelMark's martingale absorbs benign runtime jitter while catching systematic manipulation.
  3. Covert Anti-Evasion: In direct verification, the backend knows the exact expected route being checked, allowing an adaptive server to selectively fake tested routes. In SentinelMark, the backend only sees the winning route without knowing the secret scores or losing candidate, preventing selective evasion.

Thus, Table 10 shows that SentinelMark achieves 95%–97% of the throughput of an omniscient oracle, proving that confidential, covert, and elastic auditing comes at only a 3%–5% throughput cost.

We hope the new H=4,096H = 4,096 and H=8,192H = 8,192 empirical results and this clarification of the oracle baseline fully resolve your remaining concerns.

Reviewer_iMjXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SentinelMark-MoE, an online audit that checks whether a distributed mixture-of-experts backend actually executed the route selected by a trusted router. For each routing group, a trusted selector draws two feasible route assignments from a declared sampler, scores a view of each assignment with a keyed pseudorandom function under a fresh tag, and dispatches the higher-scoring assignment. After the backend commits to its execution, a trusted observer reports the executed view. The auditor then tests whether the PRF scores of the executed views retain the expected bias toward the winning candidate. The evaluation covers three MoE model families and five benchmarks, using frozen held-out route blocks. The authors also report perplexity, downstream task quality, serving overhead, and related metrics.

Critical strengths
  1. The core mechanism is simple and clear. By selecting between two feasible routes using keyed scores, the method exactly preserves the declared sampler distribution (Theorem 3.1) while introducing a detectable selection bias in the scores of the dispatched route.

  2. The paper also clearly delineates the scope of its method. It distinguishes preserving the declared sampler from preserving the native router's behavior. It points out that the pairwise selection result does not directly extend to tournaments with more than two candidates. It also proves that route-only evidence cannot certify output correctness.

Critical weaknesses
  1. No clear advantage over direct verification. The method already assumes a trusted observer. With that observer, the direct expected-view digest detects every view-changing attack in the live pilot (8/8 in all three settings, with median detection delays of 3.5, 3.5, and 193 raw decisions). Conditional LR detects 5/8, 7/8, and 8/8, with median delays of 1,232, 541, and 406. The selector metadata saving is also small: 1.147 vs. 1.209 bytes per decision at batch size 64 (Table 8), compared with the 135–570 bytes of observer data that both protocols require. Meanwhile, the keyed selection step, which direct verification does not require, accounts for a roughly 40% throughput reduction in the prototype. The paper acknowledges that this trade-off "remains unresolved" (Appendix A.4), yet it is the main reason to adopt the method.

  2. The practical claims go beyond the evidence. The abstract claims that model quality is maintained, yet Qwen HumanEval pass@1 drops from 49 to 42 and Mixtral IFEval strict accuracy from 46 to 39 (Table 2a). The only support offered is that the paired intervals over 100 records per task include zero, which is not a non-inferiority test. At load 32, MoE throughput falls to 0.59–0.63 of native. Finally, the total variation (TV) distance of 0.30 between the softened sampler and the native router (Section 4.3 and Table 7) is simply the analytic distance between the toy distributions defined in Appendix E.2. However, it is presented alongside the 50M-decision measurements as if it were a measured property of the router adapter.

Questions for authors

Which observation map was used in each row of Table 2b, and how exactly is the identity-shift fault defined?

  1. In Table 2b, the count-preserving fault has an active-challenge range of 682–857, which does not overlap the conforming range of 406–540 at all. In contrast, the reroute attack has a range of 396–540, which broadly matches the conforming range. The appendix states that paired trials reuse the same tags. Are candidate route pairs regenerated after the fault alters the hidden states?

  2. Appendix E.3 sets the conforming mean to d_n = β_2 J_n, whereas Equation (3) uses β_L ≈ 1/3. Is this merely a notation error, or did the implementation actually use 0.25? Please confirm whether the Original detector in Tables 1, 3, and 4 uses β_L.

  3. What is the end-to-end serving throughput of the direct-digest baseline under the same load-32 setup? Is there a practical deployment setting in which a trusted observer exists but a direct comparison between the expected and executed views is not possible?

  4. What non-inferiority margins would the authors consider acceptable for the generation tasks, and do the lower bounds of the paired confidence intervals satisfy those margins?

2 comments
AuthorsOpenReview ↗

Response to Reviewer iMjX

We thank Reviewer iMjX for the constructive and detailed review. We address each question below followed by our additional end-to-end serving evaluation (Table R1). Please also refer to the attached revised PDF (Appendix A.6) for detailed information.


Weakness 1: No clear advantage over direct verification

We argue that direct verification (comparing an exact route hash) requires the auditor to know the intended route for every token. This cannot apply in the cloud serving regimes we target:

  1. When an external client or regulator audits a cloud model provider, the provider cannot stream per-token routing decisions. Doing so exposes model weights, router gating logic, and private user prompt activations. SentinelMark allows auditing routing without seeing internal routing choices.
  2. In real GPU clusters, serving systems often move tokens around to balance server load. If we use an exact hash check, any normal system adjustment will trigger a false alarm. Instead, SentinelMark checks the score pattern over time. This ignores normal system noise while catches real cheating.
  3. In direct verification, the backend knows the exact route being checked, so it knows what target to fake if it tries to fool the observer. In SentinelMark, because the backend only receives the winning route without knowing the secret scores or alternative candidates, it cannot guess which tokens are safe to cheat on without breaking the score pattern.

Weakness 2: Non-Inferiority Margins and Quality

  • The reported bootstrap intervals containing zero are descriptive, not a formal non-inferiority test. Theorem 3.1 guarantees that keyed selection exactly preserves the declared sampler D\mathcal{D}. The gap to native routing is tuned via η\eta: on validation data, η=0.01\eta=0.01 limits OLMoE perplexity increase to +0.34%+0.34\% (ratio 1.0034). We clarify this in the text, and confirm that the 0.30 TV distance was only derived on a toy distribution other than measured on live models, the real model quality drop is under 0.34%.
  • Q: At load 32, MoE throughput falls to 0.59–0.63 of native
    • A: In Table 2a (Page 8), the prototype's slowdown was caused by high-level PyTorch Python hooks pausing CPU-GPU execution 32 times per token. In the same table, the dense control (Llama-2-70B, zero hooks) retained 100.2% throughput, proving the benchmark harness has zero overhead.

Question 1: Active Ranges (682–857 vs. 406–540) and Candidate Regeneration

  • Yes, candidate pairs are regenerated at each layer from the model's current hidden states.
  • Trials reuse prompts and PRF seed tags, but candidate pairs depend dynamically on layer activations. When a fault alters token 1, downstream representations change. In our audit of the 8 fault trials, 1,535 out of 1,536 candidate pairs differed from the clean run after token 1. The shifted active range (682–857 vs. 406–540) is an authentic closed-loop trajectory divergence from fault intervention, not an audit inconsistency.

Question 2: Conforming Mean Calibration (β2\beta_2 vs. βL≈1/3\beta_L \approx 1/3)

  • The code strictly uses βL≈1/3\beta_L \approx 1/3 (exact value (1−2−128)/3≈0.3333(1 - 2^{-128})/3 \approx 0.3333).
  • The notation β2\beta_2 in Appendix E.3 was a typographical index meaning 2 candidates, not the scalar 0.25. All detectors in Tables 1, 3, and 4 use βL\beta_L. We have corrected this notation in the manuscript.

Question 3

  • Q: "Is there a practical deployment setting in which a trusted observer exists but a direct comparison between the expected and executed views is not possible?"

  • A: Yes, please see response under Weakness 1.

  • Q: "What is the end-to-end serving throughput of the direct-digest baseline?"

  • A: Table R1: Throughput Comparison (OLMoE-1B-7B) | Input Batch | Buffer | SentinelMark Tok/s | Direct Verification Tok/s | Throughput Ratio | SentinelMark Bytes | Direct Bytes | |---:|---:|---:|---:|---:|---:|---:| | 1 | 64 | 2.92 | 3.09 | 0.945 | 2.45 B | 2.52 B | | 1 | 256 | 2.92 | 3.02 | 0.967 | 1.29 B | 1.26 B | | 8 | 64 | 18.85 | 19.38 | 0.973 | 2.45 B | 2.52 B | | 8 | 256 | 18.85 | 19.88 | 0.948 | 1.29 B | 1.26 B | | 32 | 64 | 54.45 | 56.52 | 0.963 | 2.45 B | 2.52 B | | 32 | 256 | 54.45 | 56.98 | 0.956 | 1.29 B | 1.26 B |

  • Table R-1 shows our updated benchmark without Python pauses, SentinelMark retains ~90% of native throughput and achieves 95%–97% of direct verification throughput.


Question 4

  • please see replies for weakness 2

Clarification on Table 2b Observation Map and Fault Definition

  • Observation Map: Every row in Table 2b strictly uses the Counter map ϕ(A)\phi(A), which records the token allocation count per expert.
  • Identity-Shift Fault: On scheduled 20% rounds, the system shifts selected expert IDs by +1(modE)+1 \pmod E. This preserves total expert call counts and top-kk distinctness while changing which experts execute.
AuthorsOpenReview ↗

Follow-up on Reviewer iMjX's Comments

Dear Reviewer iMjX,

As the discussion window progresses, we wanted to kindly follow up to see if our response and revised manuscript have addressed your questions and weaknesses. We remain at your disposal should you have any further questions, and we would deeply appreciate it if you would consider updating your score if your concerns have been resolved.

Thank you once again for your constructive review!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.