acceptodds
Under review as a conference paper at ICLR 2027

Can Strategic Ideologues Agree on Quality? Incentives on Latent Factor Models

Abstract

Community Notes ranks fact-checking notes based on ratings from ideologically diverse raters. It fits a low-dimensional model that seeks, for each note, a quality shared by all raters, separate from each rater's ideological agreement with the note. We analyze this, and related models, from a social choice perspective and discover key structural insights of these mechanisms. We leverage this structure to study both the outcomes compared to different objectives as well as incentive properties. Highlights of our results include: 1) privileging the perspective of the average rater maximizes social welfare, but is subject to simple manipulation strategies; 2) privileging the perspective of the median rater satisfies the Condorcet criterion, and is the only identifiable viewpoint robust to the same manipulations, even by coalitions; 3) more ideologically extreme agents have a larger incentive to participate.

Primary area:
learning theory
Keywords:
Community Notes, Social Choice, Strategic Incentives, Anchored Utility Model, Anchored Mechanisms

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_J1GHOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the social choice and incentive properties of latent factor models used in Community Notes. It interprets the fitted quality of an item as its evaluation from a particular ideological viewpoint, referred to as an anchor. The analysis shows that different anchors correspond to different objectives: the mean maximizes social welfare, while the median satisfies the Condorcet criterion. The paper further examines strategic misreporting and participation incentives. It establishes strategyproofness and group strategyproofness for median-anchored mechanisms under ideological spoofing, while showing that mean-anchored mechanisms are vulnerable to manipulation. The theoretical results are supported by simulations and an observational analysis of participation patterns in Community Notes data.

Critical strengths
  1. The paper establishes a useful connection between latent factor estimation and social choice. The analysis shows how the choice of anchor determines the social objective of a fitted model. The results on identifiability, re-anchoring, and strategyproofness provide a coherent theoretical account of a practical rating mechanism. In particular, Theorems 5 and 6 clarify why the median anchor is resistant to ideological spoofing under the stated conditions. This connection between statistical estimation and strategic incentives is a substantive contribution.
  2. The paper examines both rating manipulation and participation incentives. The participation analysis adds an important dimension to the strategyproofness results by showing how the composition of the rating population can change the outcome. The simulations illustrate the differences between mean- and median-anchored mechanisms, while the Community Notes analysis provides supporting observational evidence. The effort to address estimation-related confounding in the empirical analysis is also valuable.
Critical weaknesses
  1. The strategyproofness guarantees rely on restrictive assumptions that limit their applicability to deployed systems. The main results assume a one-dimensional ideological structure with strict crossing and focus on ideological spoofing. The strongest guarantees are established for exact ratings, while real Community Notes data are noisy, incomplete, and subject to more general forms of manipulation. Although Appendices F.5 and F.6 extend parts of the identification analysis to noisy and partial ratings, they do not establish the same general strategyproofness guarantees for realistic fitted models. The simulations provide supporting evidence, but the gap between the theoretical mechanism and the deployed system remains substantial.
  2. The practical advantage of median anchoring is not sufficiently established. The paper suggests that median anchoring may be preferable to the deployed mechanism, but its protection primarily concerns misreporting within a fixed rating population. Section 4.2 shows that participation itself changes the anchor, and the turnout analysis identifies cases where equilibria do not exist or where the participating population differs substantially from the full population. Median anchoring also sacrifices social welfare relative to mean anchoring in some settings. The empirical analysis supports an association between ideological distance and rating activity, but does not establish that adopting median anchoring would improve outcomes in practice. The recommendation therefore goes beyond what the current evidence directly supports.
Questions for authors
  1. Which parts of the strategyproofness results remain valid when ratings are noisy and incomplete? In particular, can the authors clarify the implications of Appendices F.5 and F.6 for manipulation of a regularized, median-anchored fitted model?
  2. How should the strategyproofness guarantee be interpreted when agents can strategically choose whether to participate? Does the participation analysis identify conditions under which median anchoring remains robust to changes in the rating population?
  3. What evidence supports the recommendation to prefer median anchoring over the deployed mechanism, given the welfare trade-off and the observational nature of the Community Notes analysis?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 3

We thank the reviewer for the careful reading. The two weaknesses and three questions all concern the distance between the theorems and the deployed system, so we answer them in that light. Given the character limit on responses, the points shared with the other reviews are addressed fully in the topic comments at the top of the forum, which we reference by their headers, and the rest in a separate comment in this thread.

Assumptions (W1). The body presents the results for the latent factor model because that is the model the deployed system fits, and it was also a simple case to analyze initially. The results, however, hold on a broader class, which App. D defines by two axioms: types are ordered, and quality is the utility of some type, and the utility difference between two items changes sign at most once along the order. Strict crossing is exactly what the median needs: the appendix gives instances where it fails, and the median loses identifiability or strategyproofness. One ordered dimension is likewise what the deployed model fits and the simplest setting in which the question can be posed. It also had no prior incentive analysis. Several dimensions require a condition on the factor geometry, since there is no strategyproof median unless the dimensions separate. We will state this as open in the Limitations (topic comment: On Clarifying Contributions and Model).

Noisy and incomplete ratings, and the regularized fit (W1, Q1). Incomplete ratings at random are already covered: Thm. 5 holds whenever the ratings stay framed once the coalition is removed (App. F.6). Since the remark was read as identification only, we will promote it to a proposition. For noise, the reviewer is right that App. F.5 gave the anchoring structure and not the strategyproofness, so we extended the theory (topic comment: On Discrete and Noisy Reports). We take the report to be a stochastic function of the utility whose expectation increases with it; the expected reports are then the exact reports of another instance of the class, and since a spoof is a draw at another type, Thm. 5 holds for the population fit. In sample the median's guarantee holds up to the estimation error of the fitted anchor, which we prove, and the reruns of Fig. 3 show that error does not move with the coalition. For the regularized fit the penalty leaves a residual that the same reruns show does not move with the coalition. Deviations that are not spoofs break the fit and are detected whenever the honest raters of a note are the majority (Lemma 9; detail in this thread). Strategic choice of which notes to rate is outside the scope of the paper, which the Limitations will state. The revision replaces App. F.5 with this subsection.

Participation (W2, Q2). Strategyproofness concerns reports, while turnout is a second channel, so the question is whether the comparison between anchors survives it. It does, in the sense that the median anchor moves less. Under the median only a band of moderates abstains, a band that narrows with the cost (Prop. 13), whereas under the mean only a few tail raters rate (Prop. 14). The turnout equilibria of Fig. 6 show the consequence: the median anchor stays within 0.13 of the population median at every cost, while the mean anchor drifts 1.2 to 2.3 into the heavier tail. Pure equilibria need not exist under either anchor, and at low cost they often do not; these statements concern the equilibria that exist, and we will say so. The condition the reviewer asks for is then that the median anchor is robust to turnout up to the abstaining band, which we will state in Sec. 4.2 and the Discussion (detail in this thread).

The recommendation (W2, Q3). We agree it reads as a blanket recommendation and will rewrite it as a decision with a trade-off: welfare under truthful reports (Fig. 4), which the mean does not reward, against robustness to consistent exaggeration and to turnout (topic comment: On the Recommendation of the Median Anchor). As for evidence, we can offer two kinds. On the public Community Notes data, switching the deployed anchor to the median changes the shown note on about 15% of the posts that have one, in the predicted direction, so the choice has consequences today (topic comment: On Applicability and Connection to Community Notes). Only an experiment or a deployment can show the incentive response, which is where the welfare trade-off plays out. We will state that as the boundary of the claim.

We appreciate the questions, which sharpened our understanding of where the theory needed to be extended, and would welcome any further pointers to places where the argument could be strengthened.

AuthorsOpenReview ↗

Response to Reviewer 3 (Additional Details)

On Incomplete Ratings and Other Manipulations (W1, Q1)

Incomplete ratings at random are covered: the remark after the framed-ratings lemma in App. F.6 gives Thm. 5 whenever the ratings stay framed once the coalition is removed. Its one exception is a note rated by the coalition alone, which the deployed minimum-rating filters discard. Since the remark was read as identification only, we will promote it to a proposition.

As for other manipulations, any deviation that is not a spoof breaks the fit and could be detected (Lemma 9). When the deviating accounts are fewer than half of a note's raters, they are also identified. This is the class the deployed screens target; the second stage of the attack in Selvam et al. is an example. Noise is treated separately in the comment "On Discrete and Noisy Reports". We would also make it clear that strategic partial ratings are outside the scope of our work but are a promising direction.

On participation (W2, Q2)

Strategyproofness is a statement about reports, which we analyze under spoofing. Turnout, however, is a second channel, which we analyze in Sec. 4.2. We first establish why a more ideologically extreme participant derives a greater benefit from rating than from abstaining, and then analyze the equilibria under different anchors. Under the median, the raters who abstain form a band of moderates, which narrows as the cost falls (Prop. 13). Under the mean, by contrast, a rater's influence shrinks as more agents rate, so only a few raters in the tails rate (Prop. 14). The turnout equilibria of Fig. 6 show the consequence: the median anchor stays within 0.13 of the population median at every cost, while the mean anchor drifts 1.2 to 2.3 into the heavier tail. Pure equilibria need not exist under either anchor, and at low cost they often do not, so these statements concern the equilibria that exist. Within them, the comparison survives turnout in a precise sense: the median anchor moves less, up to the abstaining band. We will say so in Sec. 4.2 and the Discussion.

Thank you for the detailed rebuttal and clarifications. I appreciate the authors' efforts to address my concerns, but I will maintain my initial score.

Dear reviewer,

Thank you again for your response and helpful feedback. Please let us know if you have any other questions.

Kind regards, authors

Reviewer_vqMMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies latent factor utility models inspired by Community Notes, where each user has an ideological type and their utility for a note is the sum of its true global quality and their own ideological type * the ideological leaning of the note. They study several properties and provide several insights: how different "type anchoring" mechanisms relate to societal objectives, strategy-proofness, participation turnout equilibria etc.

Critical strengths

Strengths:

  1. The authors have an illustrative model of CN that is a nice abstraction of the key CN model (although it may also have some deviations from CN that make applicability of the insights to the platform unclear, see Weakness 1)
  2. They have several nice insights that arise out of the simplified model, e.g., the correspondence between type anchoring and the social objectives, that OQ is not identifiable, that the median is the only identifiable strategyproof anchor, the way that anchors will affect participation rates.
Critical weaknesses

Weaknesses:

  1. In practice, reports are usually something discrete categories like Helpful / Not Helpful, whereas the paper defines a report as being any real number, with the report being truthful if the report’s value exactly matches the user’s utility for the note. How do the inferences that are drawn with the authors’ model apply in the real setting where users’ reports are discrete categories?
  2. In the discussion the authors state: “According to our results, we suggest that anchoring at the median is likely superior to the currently deployed mechanisms” I think this might be a strong inference based only on their results in the paper. The main benefit of the median that they show is its resistance to strategic manipulation (particularly “spoofing” where the user reports a utility that arises from a different user type than their true type). But if the “reports” are discrete categories like in practice, say +1 : Helpful / -1 : Not Helpful, then I would expect the mean to also have this kind of strategyproofness? By the same logic for the median, the only alternative that a user can deviate to will push the mean away from its own type. (This is intuition, I might be wrong and it would be good to hear the author’s thoughts.)
  3. The work is missing a key citation about strategic manipulation in CN that was done in collab with the X CN team: Selvam et al (2026) “Gaming Consensus: Coordinated Manipulation in Crowdsourced Fact-Checking” Their strategy for an attack, is as you might expect, a two-stage approach where in the first stage the attacker tries to create accounts with diverse user types, and then finally for the target note that they care about, once they vote with these diverse types, they can cause the note to appear. Though in some ways, this work reveals how difficult it is to strategically manipulate CN in practice. In Stage 1, each of their dummy accounts rates 100 random notes to establish its factor before their Stage 2 attack. Many simple mitigations implemented in X CN (that were not used in the evaluation of the attack) already significantly reduce the potential for this style of attack. So in general, coming back to the author’s recommendation that the median is likely to be superior to what’s currently deployed— I’m skeptical of the strength of this claim, given that CN is already hard to strategically manipulate, and it’s not clear whether the benefits of the median anchoring still hold up with a more realistic model of reporting on CN (see Weakness 2)
  4. The paper does not do any experiments on the public CN data (they only use it to look at participation rates on CN)! It’s such a treasure that the X CN data is open source, it’s a shame it’s not used. Although this paper is primarily theoretical, the argument could become much stronger if they actually ran the open source CN algorithm on the CN data as a baseline. They could try doing a version of the mean anchoring and median anchoring, and also explore how manipulable the default CN algo actually is, compared to the anchored versions proposed by the authors.
  5. “The theory is also consistent with Community Notes data: on US-politics notes, the fifth of raters farthest from the median rate about 1.5 times as much as the fifth nearest to it (Appendix I.6).” — I guess, but also it could simply just be that more extreme political users are more motivated to rate (regardless of what the aggregation mechanism is). I don't feel like this says much.
  6. The paper’s text could be significantly improved for clarity and flow. Please do so!
  7. The citation that is referenced the most often is an anonymous citation that cannot be accessed for comparison.

Minor:

  • The dashed lines in Figure 1 indicate the rater’s top note? Would be useful to specify in the figure caption
  • Thm 6 not sure why “anonymous” and “efficient” are only defined in a footnote, would recommend bringing out into the main text beforehand with more explanation
  • I don’t think “shortfall” is defined clearly ?
Questions for authors

Overall, I enjoyed the paper. But I think the paper may overclaim applicability to CN in the wild. If the authors can convince me of the applicability of their results to actual CN systems, I would raise my rating. Possible ways the authors could do this include: (1) Explain how their theoretical results change when reports are discrete categories as in the real implementations, e.g., -1 : Not Helpful and +1 : Helpful, (2) Run experiments with the open source X CN data and algorithm, (3) Hedge claims of applicability to CN where the model provides only a simplified account

4 comments
AuthorsOpenReview ↗

Response to Reviewer 2

We thank the reviewer for the detailed reading, and for stating which concerns could be addressed first. We take them as: (1) saying how the results change under discrete reports, (2) running the open-source algorithm on the public data, and (3) hedging where the model simplifies. We address all three below. The full treatment of shared points is in the topic comments at the top of the forum, referenced by header.

Discrete reports (W1, and the intuition in W2). The reviewer's intuition is right for a single rater, since one account's reports saturate. However, a coalition's displacement grows with its size, which is why coordinated accounts are the realistic threat and why the guarantee in the paper is group strategyproofness (Thm. 5). In addition, we ran the spoofing experiment of Fig. 3 with binary and three-level reports and found the results the theory predicts, and we added a short extension of the theory that covers stochastic reports in general (topic comment: On Discrete and Noisy Reports).

Community Notes data (W4). We ran the open-source core scorer on the public ratings and imposed the median and mean anchors (topic comment: On Applicability and Connection to Community Notes). Switching to the median changes the shown note on about 15% of the posts that have one, in the direction Thm. 4 predicts. There is no ground truth, so this shows the ranking changes; whether it improves depends on the raters' incentive response, which only an experiment or a deployment can show. We also checked that the core's decisions are the deployed ones: 98% of the notes the core rates helpful keep that status after every later scorer and rule. We did not inject spoofers into the replica, because manipulability of the full pipeline is the question Selvam et al. answer, whereas ours is which anchor the pipeline ranks from. The simulations of Fig. 3 and their reruns on discrete reports make the comparison the reviewer asks for: the default anchor moves with the coalition and the median does not.

The recommendation (W2, W3). We agree the discussion could read as a blanket recommendation, so we will edit it to make the decision and its trade-offs salient, and we will clarify in the introduction and the Discussion which results concern the core model, which concern the full pipeline, and where the model simplifies. We also want to say why we think the median is a reasonable default. Under truthful reports, the median gives up some welfare (Fig. 4), but truthful reporting is not what the mean rewards; in exchange, the median is robust to consistent exaggeration, which passes every consistency check of the fit (Lemma 8, Thm. 5), and to turnout (Props. 13, 14). The reviewer's W3 reads Selvam et al., which we had missed and will cite, as showing that Community Notes is already hard to manipulate. We read their result differently (topic comment: On the Recommendation of the Median Anchor): their attack targets a single note, which a fit detects whenever the honest raters of that note are the majority (Lemma 9), and that is why it needs many accounts and why the mitigations address it. The anchor channel moves when raters report consistently with their leaning, and each such rating is the honest rating of some type, so single-note checks do not see it. "Hard to manipulate" therefore concerns single notes, while the recommendation concerns the anchor, which the deployed system sets implicitly.

Participation evidence (W5). The reviewer is right that the gradient admits another explanation, that extreme raters are more motivated whatever the mechanism, so we will say "consistent with" rather than "support". Seen from the theory, under the deployed anchor that pattern is precisely what the mechanism rewards (Prop. 14), while under a median anchor it would not be (Prop. 13). So the data cannot separate motivation from reward (detail in this thread).

Companion paper (W7). We wrote this paper to be largely self-contained. We cite the companion for context and to avoid claiming its contributions twice; since submission it has been accepted at NeurIPS 2026, so the final version will cite it by name (detail in this thread).

Presentation (W6, minors). We appreciate the pointers the reviewer has given to improve the clarity of the work and will implement them: the caption of Fig. 1 will say what the dashed lines mark, anonymity and efficiency move from the footnote of Thm. 6 into the text before it, and shortfall is defined where it first appears, as the gap between the objective a selection attains and the best attainable. The revision also adds a contributions paragraph with pointers (topic comment: On Clarifying Contributions and Model). If there are other places the reviewer feels could be strengthened, we would welcome them.

One question back: does the distinction between the single-note channel, which Selvam et al. attack, and the safeguards screen and the anchor channel match the reviewer's reading of the deployed mitigations?

AuthorsOpenReview ↗

Response to Reviewer 2 (Additional Details)

On the participation evidence (W5)

The reviewer is right that the Community Notes gradient admits another explanation: extreme raters may simply be more motivated to rate, whatever the mechanism. We therefore agree that the gradient is consistent with the theory rather than a test of it. We did control the confound we could. The scorer's penalty shrinks the factors of raters with few ratings toward zero, so heavy raters look farther from the center for mechanical reasons alone; measured in one window, that effect by itself gives a correlation of 0.43 between distance and activity. We removed it by estimating every rater's factor from the same number of ratings in one window and counting their ratings in the next, and the gradient holds at the deployed anchor, the median and the mean. Seen from the theory, though, the two explanations make the same prediction. Under the deployed anchor, strong raters rating more is precisely what the mechanism rewards (Prop. 14), whereas under a median anchor it would not be (Prop. 13). The data cannot separate motivation from reward, so we will change "support" to "are consistent with" in the introduction.

On the companion paper (W7)

We wrote this paper to be self-contained. The companion paper is cited for context, to avoid claiming its contributions twice, and for one fact proved there: that L2-regularized biased matrix factorization anchors at a shrunk fraction of the mean type. We use that fact in two places, the sentence in Sec. 4.1 on why a spoof is rewarded under the deployed estimator and the quality-sensitive variant in App. F.4. In both, the experiments also measure the realized anchor directly (App. I.3). Since submission, the companion has been accepted at NeurIPS 2026, so we will cite it by name in the final version. We can provide an anonymized copy as supplementary material if the AC approves and the reviewers find it helpful.

Dear reviewer,

Thank you again for your helpful feedback. Please let us know if you have any other questions, or if we have adequately addressed your concerns.

Kind regards, Authors

I thank the authors for their detailed rebuttal. I have raised my score to a 3 - weak accept.

Reviewer_GiJCOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
3(+2)
Read review
Paper summary

The paper studies the rating aggregation mechanisms for the Community Notes of X from the angle of social choice. The work uses a latent factor model to predict the utility of agents by taking items’ quality, agents’ types, and items’ leaning. Theoretically, the paper shows the properties of mechanisms anchoring different rater types, including efficiency properties such as social welfare maximisation and the Condorcet property, and the manipulation property, i.e., the strategyproofness. Then, the work shows that raters with extreme ideologies have stronger incentives of participation, and thus, recommends using median-based mechanisms.

Critical strengths
  1. The work’s angle that studies real-world rating systems through the scope of social choice is worth exploring.
Critical weaknesses
  1. The alignment of the established model and the practice is unclear. The work does not provide enough explanation of the rationale for the model setting, e.g., agent types, item leaning, the linear model of ideology, and so on. Therefore, it is not clear whether the model makes sense in the real world. I would prefer that the authors add detailed explanation to the model settings.
  2. Many of the statements and settings are confusing. For instance, the mechanism can be two types, namely, one outputs an ordinal ranking of items and the other is more like a multi-winner election. These different mechanisms may need different properties to characterise, e.g., the classic Cordorcet property does not fit the analysis of multi-winner voting mechanisms. Many other statements are also ambiguous. For example, in Theorem 1, “no two items tie in quality, in total utility, or beat each other”.
  3. I have not enjoyed reading the paper due to the presentation. The writing is not formal, leaving many English problems and unclear statements. I do not think the paper is ready for publication.
  4. Another confusing point is that the paper shows several figures which seem to be experimental results in the main text. However, no details are provided in the main text.
Questions for authors

See the weakness part.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 1

We thank the reviewer for their reading. We would like our work to be accessible to the ICLR community, and we understand that there may be places where our contributions or approach could use more clarity. We would therefore be grateful for a pointer to the statements or proofs the reviewer considers unsound. To add, while addressing all reviews we have also added material that may speak to the concern about soundness and applicability: runs of the open-source Community Notes algorithm on the public data, the spoofing experiment repeated on discrete reports, and an extension of the theory to stochastic reports, each in a topic comment at the top of the forum. We address the reviewer's comments below.

On the model (W1). The form is the one the deployed Community Notes scorer fits, a quality shared by all raters plus the rater's type times the note's leaning, and it is also a valence ideal-point model from political science. The topic comment "On Clarifying Contributions and Model" provides the reasons for the model, the broader class to which the results hold, and the rationale for one dimension. In the revision, Sec. 2 will say so.

On ranking versus set, and Thm. 1 (W2). A mechanism outputs a ranking, of which the selection is the top k. The reviewer is right that the two can need different properties in general. Here they coincide: under strict crossing, the majority relation is the median type's ranking, so it is a linear order, and the selected set is the top k of that order, which makes the Condorcet criterion well defined for both. The condition quoted from Thm. 1 is a genericity assumption: no two items tie in quality, in total utility, or in the majority relation, so each of the three rankings is strict. We will name it, state it before the theorem, and add a remark on ranking versus selection.

On presentation (W3, W4). The figures in the body summarize experiments whose full setup is in App. I, so Secs. 3 and 4 will contain a setup sentence that points there. We will also do a pass for clarity. If there are sections where the presentation most hampers the argument, a pointer would help us revise those first.

On novelty. We would like to point to the topic comment "On Clarifying Contributions and Model". To add, relative to Moulin's characterization of strategyproof rules on single-peaked preferences, which Sec. 4 cites, the preferences here are latent, so the anchor must be identifiable from the ratings. That requirement cuts Moulin's family of strategyproof generalized medians down to one: among the identifiable anchors, only the median is strategyproof (Thm. 6). We have also established implications on Community Notes: on the public data, switching the deployed anchor to the median changes the shown note on about 15% of the posts that have one (topic comment: On Applicability and Connection to Community Notes).

Dear reviewer,

Thank you again for your helpful feedback. Please let us know if you have any other questions, or if we have adequately addressed your concerns.

Kind regards, authors

I appreciate the detailed answers provided by the authors. My original concerns mainly included two points:

(1) The rationale for the model establishment, e.g., the setting of the utility function. After further investigation, combined with the replies from the authors, I found that the setting originates from existing works, though I’m not fully convinced that the raters would behave like this. This setup serves the theory development well, but I would not expect a high practical impact of the results. So, my main concern is the model assumption and the practical applicability (which I grouped under the soundness issue). The analytical results are safe based on the behavioural assumption.

(2) The presentation issues: (i) terms used without definition or explanation, and (ii) experimental demonstrations without introducing the settings in the main text. I appreciate the authors’ commitment to revising the content. However, I would not say that showing experimental figures with only brief captions is a decent way to present the experimental results. The main text should be self-contained.

Based on the authors’ reply, I would like to raise my score to 2, considering some of my main concerns still remain unresolved.

Thank you for the reconsideration and for stating the remaining concerns so clearly. We are glad the analysis itself is no longer in question.

Regarding the behavioral assumption, we understand it as having two parts. The first is about what raters value. It is that the model is well specified, meaning that raters' utilities have the form the mechanism fits. This is an assumption the deployed mechanism already makes, since it is what lets the platform read the fitted intercept as helpfulness. To note, our results also need less than the linear form, only ordered types, the existence of a 'neutral' type, and strict-crossing utilities (App. D). With no further assumption, any such mechanism ranks from an anchor, and the anchor decides the social objective (Thms. 2 and 4). The second part is about how raters act, and it is ours. It is that raters respond to what the mechanism rewards. Incentive results depend on it. We think observational evidence is at least consistent with it: on the public data, raters farther from the center rate more, which is the pattern Prop. 14 predicts under the deployed anchor (App. I.6). We agree this is not a full test, since it admits other explanations, and the revision of the paper will state the untested response as a boundary of our claims.

Regarding practical impact, we would like to highlight that Community Notes and similar mechanisms are deployed on platforms with very large user bases, yet, to our knowledge, formal analyses of these mechanisms remain rare. Thanks to the reviewers' suggestions, the additional tests we ran on the public data show that the anchor's impact is not negligible. If the deployed anchor were changed to the median, the shown note would change on ~15% of the posts that have one (topic comment: On Applicability and Connection to Community Notes). We therefore see the choice of anchor, which the deployed system makes implicitly, as a decision worth understanding even where the model simplifies.

Regarding presentation, we agree. The paper is primarily theoretical, and in giving space to those results we left the experiments with too little introduction. We will revise the main text so that the experimental approach and the setting of each figure are introduced before the results are shown, as far as the page limit allows, with the full details remaining in App. I. Terms will also be defined at first use.

Thank you again for your follow-up.

Kind regards, authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.