acceptodds
Desk rejected at ICLR 2027

Hypergraph Pattern Machine: Compositional Tokenization for Higher-Order Interactions

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Hypergraphs model higher-order relations that drive real-world decisions, from drug prescriptions to recommendations. A central structural signal in such data, beyond what pairwise relations can express, is interaction compositionality: whether a higher-order relation is compositional, emergent, or inhibitory with respect to its subsets, i.e., whether a set and its one-element-smaller subset are both observed, only the larger one is observed, or only the smaller one is observed. These patterns are read directly from the observed data. In adverse-event reports, for example, they distinguish drug sets whose reported association persists when a drug is added, appears only for the full set, or is absent after adding a drug; these are reporting patterns, not pharmacological mechanisms. However, existing hypergraph learning methods, which propagate messages over observed hyperedges, do not represent unobserved subsets explicitly and leave this signal implicit. To this end, we propose the Hypergraph Pattern Machine (HGPM), which makes the compositional pattern of subsets an explicit part of the input. It tokenizes observed and unobserved subsets around each target, organizes them in an inclusion DAG, and pretrains an inclusion-aware Transformer with masked reconstruction. On ten hypergraph benchmarks, HGPM matches or exceeds state-of-the-art methods. A qualitative case study on adverse-event reports further illustrates how HGPM separates candidate drugs that are close in feature space but have different recorded patterns.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Hypergraph, Higher-Order Interactions, Substructure Tokenization, Drug Interaction Prediction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_x2pzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper turns subsets around a target node, both observed and unobserved, into tokens arranged in an inclusion DAG. A Transformer with inclusion and role biases encodes them, and is pretrained with masked reconstruction against an EMA teacher. Experiments cover eight node classification benchmarks and two adverse-event datasets.

Critical strengths
  1. Representing absent subsets explicitly is an interesting idea. It gives the model access to patterns that standard hypergraph message passing can miss.

  2. The inclusion DAG fits the subset relationships well, and the results across several datasets suggest that the representation is useful.

Critical weaknesses
  1. The definition of existence in Section 3 does not match the tokenizer. Section 4.1 gives sampled sub tokens existence 1 even when they are not observed hyperedges. The same subset can instead become a neg token with existence 0 if it is not sampled as a sub token. Its label therefore depends on the sampler. This makes it unclear how faithfully the input represents the COMP/EMER/INHIB patterns defined in Section 3.

  2. Theorem 3 does not explain the benefit of inclusion-aware attention. Its separating construction sets the structural biases and attention contributions to zero; token-type counts alone distinguish the two hypergraphs. This supports the value of subset tokenization, but leaves the role of inclusion-aware attention unresolved.

Questions for authors
  1. Why do unobserved sub-hyperedges get existence 1? How does this fit with Section 3, and with the same subset getting different labels across views?
  2. What can inclusion-aware attention tell apart that mean pooling over the same token attributes cannot? Does that hold for the real bounded tokenizer, not just the idealized one?
4 comments

W1 and Q1. Existence of sub tokens

"The definition of existence in Section 3 does not match the tokenizer. Section 4.1 gives sampled sub tokens existence 1 even when they are not observed hyperedges. The same subset can instead become a neg token with existence 0 if it is not sampled as a sub token. Its label therefore depends on the sampler."

Q1: "Why do unobserved sub-hyperedges get existence 1? How does this fit with Section 3, and with the same subset getting different labels across views?"

  • Why do unobserved sub-hyperedges get existence 1? They do not in the implemented tokenizer; the statement in Sec. 4.1 and App. C.1 is corrected as below and in the updated draft. The implementation and all reported results are unchanged. Every non-center token receives the indicator of Sec. 3, 1S=1[S∈E]\mathbf{1}_S=\mathbf{1}[S\in\mathcal{E}], where E\mathcal{E} is the set of hyperedges visible to the model, whatever its source tag. In an audit of the tokenizer's outputs on the eight node-classification benchmarks (229,518 views, before pretraining masking), 99.29–100% of sub tokens outside Congress have existence 0; the share is 73.09% on Congress. The others are fragments that are themselves hyperedges (first table below).
  • How does this fit with Section 3? For each retained adjacent pair with at least one observed endpoint, the implemented membership rule gives the COMP/EMER/INHIB label defined there. Pairs with state (0,0)(0,0) are outside the three regimes. Sampling determines which candidate subsets and relations are retained, not their membership values. Within the original token budget, the sampler first keeps both endpoints of available COMP, EMER and INHIB adjacent-order inclusion relations and then fills the remaining positions, admitting neg tokens only into positions that the obs/sub pool leaves free. All tokens remain subject to the original per-order caps and source constraints, so retaining all three regimes in every view is not guaranteed; the second table below reports the resulting coverage.
  • The same subset with different labels across views? The source tag records provenance only. A sub token is a fragment of an observed hyperedge. A neg token is obtained by dropping, adding or swapping one member of an observed hyperedge or sub token, and is kept only if it is not a hyperedge. A subset reached by either route receives the same existence. The audit found no membership errors or cross-view existence conflicts on any of the eight benchmarks (first table below).

Existence of sub tokens in the implemented tokenizer's outputs. Counts are token occurrences across views; existence is checked against the visible hyperedge set. In the last column, a subset is identified by its node set and compared across the views in which it occurs within the same dataset.

Datasetsub tokensExistence 0Existence 1 (the fragment is itself a hyperedge)Share with existence 0Subsets with conflicting existence across views
Citeseer30,68930,52416599.46%0
Pubmed438,831435,7223,10999.29%0
Cora-CA77,64477,35628899.63%0
DBLP-CA1,762,5611,759,0133,54899.80%0
Congress31,00922,6638,34673.09%0
Senate66,56966,5690100.00%0
Walmart234,225233,65756899.76%0
House250,322250,3139100.00%0

Retained adjacent-order inclusion edges and view coverage. Each cell gives edge count (percentage of all views containing at least one such edge). Both endpoints have order at least 2. Each smaller-to-larger pair is counted once per view, before masking; center/readout edges and cross-view pairs are excluded. A pair occurring in several views is counted in each, so these are not globally distinct-pair counts. The percentages are view coverage, not edge shares, and need not sum to 100%.

DatasetCOMP (1,1)(1,1)EMER (0,1)(0,1)INHIB (1,0)(1,0)(0,0)(0,0)
Citeseer414 (3.6%)8,816 (29.9%)4,871 (42.0%)46,124 (31.6%)
Pubmed2,908 (4.1%)59,500 (17.3%)19,288 (18.8%)419,525 (18.4%)
Cora-CA310 (5.2%)9,958 (53.7%)5,257 (65.8%)108,054 (77.6%)
DBLP-CA6,072 (6.3%)211,145 (68.9%)109,283 (82.7%)2,261,516 (90.0%)
Congress4,982 (88.2%)4,508 (100%)3,909 (99.9%)1,244 (10.9%)
Senate8 (1.4%)2,998 (100%)2,248 (100%)49,956 (100%)
Walmart607 (0.7%)34,721 (34.2%)28,265 (31.2%)138,252 (66.1%)
House30 (0.6%)805 (6.2%)180 (6.2%)190,509 (100%)

The idealized tokenizer of App. B.3 (Lemma 3) already follows this membership rule: it gives 1S=0\mathbf{1}_S=0 to the unobserved 2-subsets of observed hyperedges.

W2 and Q2. Theorem 3 and inclusion-aware attention

W2.1. What Theorem 3 shows

"Theorem 3 does not explain the benefit of inclusion-aware attention. Its separating construction sets the structural biases and attention contributions to zero; token-type counts alone distinguish the two hypergraphs."

We agree with this reading of Theorem 3. Its witness sets the structural biases and the attention contribution to zero and separates the two hypergraphs through the number of tokens of each type, as Remark 6 states. It establishes the value of subset tokenization; the role of inclusion-aware attention is shown by the example in W2.2.

W2.2 and Q2. What inclusion-aware attention can tell apart

Q2: "What can inclusion-aware attention tell apart that mean pooling over the same token attributes cannot?"

Two views whose token sequences are identical and whose inclusion edges differ, under the same tokenizer as Theorem 3. Take constant node features, H={c,a,b,d}H=\{c,a,b,d\}, and

\qquad \mathcal{E}_B=\{\{c,a\},\{c,b,d\},H\}.$$ For center $c$, the tokenizer of Lemma 3 keeps every observed hyperedge that contains $c$ and all of their 2-subsets that contain $c$, with existence given by membership. In both hypergraphs this yields the center token and the five tokens below, where $T$ is the observed triple ($\{c,a,d\}$ in A, $\{c,b,d\}$ in B). |Token|Order|Existence|Adjacent superset in A|Adjacent superset in B| |---|---:|---:|---|---| |$P=\{c,a\}$|2|1|$T$ (COMP)|-| |$Q=\{c,b\}$|2|0|-|$T$ (EMER)| |$R=\{c,d\}$|2|0|$T$ (EMER)|$T$ (EMER)| |$T$|3|1|$H$ (COMP)|$H$ (COMP)| |$H$|4|1|-|-| The two token sequences are identical in feature, order, existence, source and position: $P$, $T$ and $H$ are observed hyperedges in both hypergraphs, and $Q$ and $R$ are unobserved 2-subsets of $H$ in both, so no token's source depends on the hypergraph. The token-type counts used by the witness of Theorem 3 therefore coincide, and any encoder that reads only the tokens returns the same output for A and B, for every choice of weights; this includes mean pooling and a Transformer without structural biases. The inclusion edges differ. In A the triple extends the observed pair $P$ (COMP) and the unobserved pair $R$ (EMER). In B it extends only the unobserved pairs $Q$ and $R$ (two EMER edges), and the observed pair $P$ has no adjacent superset. The direction bias $b^{\mathrm{dir}}$ is defined on these edges. With a nonzero $b^{\mathrm{dir}}$ and every other bias at zero, an explicit weight setting gives different representations for A and B: only the token $T$ changes, because it attends to one observed and one unobserved pair in A and to two unobserved pairs in B. By Lemma 5, HGPM then separates A and B for almost all weights. ### W2.3 and Q2. The bounded tokenizer > Q2: "Does that hold for the real bounded tokenizer, not just the idealized one?" The example needs no retention assumption under the tokenizer of Lemma 3. Under the bounded sampler it holds for views that retain these five tokens, which the sampler does not guarantee in general. The coverage table in W1 reports coverage under the actual sampler, while Table 2(c) compares encoders on its outputs. The empirical counterpart with the actual tokenizer is Table 2(c). The inclusion-aware encoder reaches 86.03 / 80.42 (homophilic / heterophilic), against 84.78 / 77.99 for a vanilla Transformer on the same tokens and 83.29 / 76.29 for mean pooling. On drug edge classification, the task is to predict the label of a (drug set, adverse event) record, and HGPM additionally reads subset-state evidence: for each subset of the queried drug set among its tokens, the state (positive, negative or unknown) of that subset for the queried adverse event (AE) in the training records, with the queried set's own state masked. With tokens, subset-state evidence and pretraining matched, the inclusion-aware encoder is 1.3 / 1.1 F1 points (HODDI / JADER) above the vanilla Transformer.

Dear Reviewer,

As the discussion period is drawing to a close, we would like to check in on whether our response has resolved your concerns. We are happy to answer any further questions. If our response has addressed your concerns, we would be grateful if you would consider raising your rating. Thank you again for your time and thoughtful feedback.

Best Regards, Authors

Dear Reviewer,

With about 10 hours left in the discussion period, we would be happy to address any remaining questions or concerns you may have. Thank you again for your time.

Best regards,
The Authors

Reviewer_oTh1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper claims that existing hypergraph models only pass messages along the groups that exist in the data, so they never see which smaller or larger groups are missing. The idea is that when one item is added to a group, the larger group may still be observed, may appear only as a whole, or may disappear. The proposed HGPM turns this into model input. Around each node it lists nearby subsets, both observed and unobserved, links them by one-element-larger relations, marks each as present or absent, and feeds them to a Transformer that is pre-trained by masking subsets.

Critical strengths
  1. It asks a fresh, well-motivated question: should absent subsets be a first-class input to hypergraph learning?
  2. Theorem 1 gives a clean formal footing: the three labels are a discrete difference of the hyperedge indicator, and simplicial complexes are exactly the "no-emergent" case.
Critical weaknesses
  1. There is a problem with tokenizer inconsistency. Subsets sampled from observed hyperedges (sub tokens) are assigned existence = 1, even when they are not observed hyperedges themselves. This can turn genuinely emergent transitions into compositional ones. The observed-first token budget can also exclude negative tokens when the positive pool fills the cap. Moreover, the theoretical analysis relies on an idealized tokenizer rather than the actual sampling procedure.

  2. There is a missing preptrain control in comparisions. HGPM performs notably worse without pretraining, and the best objective is generic EMA reconstruction. Removing missing-subset tokens causes only a small drop, leaving the contribution unclear.

  3. HGPM uses training labels of the query subsets, while the baselines do not receive the same query-specific information. A matched-input comparison and temporal split would help clarify the reported gains and cross-order generalization. The FOLFOX case study is informative but limited to one example, with predicted regimes closely tied to binary labels.

Questions for authors
  1. Can the authors explain what is the difference between a negative unknown-sub-tokens (meaning that these sub-tokens never form a hyperedge?) and a positive unknown-sub-tokens, when both are assigned existence = 1?
  2. Can the author design a pretraining control for a fair comparison?
7 comments

W1 and Q1. Tokenizer

W1.1 and Q1. Existence of sub tokens

"Subsets sampled from observed hyperedges (sub tokens) are assigned existence = 1, even when they are not observed hyperedges themselves. This can turn genuinely emergent transitions into compositional ones."

Q1: "Can the authors explain what is the difference between a negative unknown-sub-tokens (meaning that these sub-tokens never form a hyperedge?) and a positive unknown-sub-tokens, when both are assigned existence = 1?"

Sec. 4.1 and App. C.1 give every sub token existence 1. Under that description, an unobserved fragment of an observed hyperedge would be presented as observed, and a genuinely emergent transition would be read as compositional. We correct the description accordingly, as below and in the updated draft. The correction concerns the text only; the implementation and all reported results are unchanged.

In the implemented tokenizer, every non-center token receives the indicator of Sec. 3,

1S=1[S∈E],\mathbf{1}_S=\mathbf{1}[S\in\mathcal{E}],

where E\mathcal{E} is the set of hyperedges visible to the model. The rule is the same for every source. The source tag τ(S)∈{obs,sub,neg}\tau(S)\in\{\mathrm{obs},\mathrm{sub},\mathrm{neg}\} records how a token was generated and does not affect its existence. Emergent transitions therefore stay emergent: if {A,B,C}\{A,B,C\} is a hyperedge and {A,C}\{A,C\} is not, {A,C}\{A,C\} has existence 0 whether it was generated as a sub token or by perturbation, and the edge {A,C}→{A,B,C}\{A,C\}\to\{A,B,C\} has the state pair (0,1)(0,1), which is EMER as defined in Sec. 3, not COMP.

We audited the implemented tokenizer's outputs on the eight node-classification benchmarks, covering 229,518 views, before pretraining masking. In the audit table below, 99.29–100% of sub tokens outside Congress have existence 0; the share is 73.09% on Congress. The remaining fragments are themselves observed hyperedges, and no cross-view existence conflicts were found.

Existence of sub tokens in the implemented tokenizer's outputs. Counts are token occurrences across views; existence is checked against the visible hyperedge set. In the last column, a subset is identified by its node set and compared across the views in which it occurs within the same dataset.

Datasetsub tokensExistence 0Existence 1 (the fragment is itself a hyperedge)Share with existence 0Subsets with conflicting existence across views
Citeseer30,68930,52416599.46%0
Pubmed438,831435,7223,10999.29%0
Cora-CA77,64477,35628899.63%0
DBLP-CA1,762,5611,759,0133,54899.80%0
Congress31,00922,6638,34673.09%0
Senate66,56966,5690100.00%0
Walmart234,225233,65756899.76%0
House250,322250,3139100.00%0

To answer Q1 directly: sub and neg differ in how the subset relates to the observed hyperedge it was generated from, not in existence. A sub token is contained in an observed hyperedge. A neg token is obtained by dropping, adding or swapping one member, so it can be a subset, a superset or a sibling of its source, and it is kept only if it is not a hyperedge. A neg token always has existence 0. A sub token has existence 0 unless the fragment is itself a hyperedge.

W1.2. Token budget

"The observed-first token budget can also exclude negative tokens when the positive pool fills the cap."

The reviewer's reading is correct: perturbation (neg) tokens enter an order only through positions that the pool of obs and sub tokens leaves free, so an order whose pool reaches the cap KoK_o admits none. What App. C.1 describes incompletely is how positions are filled within the cap. The implemented sampler works in two steps.

  1. Relation endpoints. It first selects, greedily and subject to the per-order caps and the pool order above, pairs of tokens at adjacent orders that form a COMP, EMER or INHIB relation, and keeps both endpoints.
  2. Remaining positions. It then fills what is left of each order as App. C.1 states: from the obs/sub pool first, and from neg candidates only if positions remain.

Endpoint priority therefore decides which obs and sub tokens are kept, so that the retained tokens form relations with one another. It does not admit a neg token into a full order. The unobserved endpoints it keeps at such an order are sub fragments, which under the membership rule of W1.1 are unobserved in 99.29–100% of cases outside Congress (audit table in W1.1). The cap thus limits one source of unobserved subsets and does not remove them from the view. The selection uses the hyperedge set only and no node labels.

Direct audit. We counted these cases on five datasets, shown in table below. A slot is one order of one view. The cap is reached in 5.4–81.0% of slots: rarely on Citeseer and Pubmed, and in a third or more of the slots on the other three. No slot at the cap holds a perturbation token. Even so, 87.1–99.2% of these slots still hold at least one existence-0 token, supplied by the sub fragments; in the remaining 0.8–12.9%, every token retained at that order is observed. Slots that are at the cap and hold no existence-0 token therefore make up about 0.2% (Pubmed) to 10.4% (Congress) of all slots.

Where the cap binds often, this fill-in role is one reason why removing perturbation tokens costs little (0.22 / 0.81 points on average, Table 2(a)): they are absent from every slot at the cap, and the unobserved subsets the encoder reads there are sub fragments.

Slots at the cap and the unobserved tokens they keep. The last two columns are shares of the slots at the cap.

DatasetSlots at the capWith no neg tokenWith an existence-0 token
Citeseer9.7%100%96.2%
Pubmed5.4%100%96.8%
DBLP-CA34.7%100%99.2%
Walmart67.4%100%90.9%
Congress81.0%100%87.1%

W1.3. Idealized tokenizer in the theory

"Moreover, the theoretical analysis relies on an idealized tokenizer rather than the actual sampling procedure."

We agree. Theorem 3 is proved for the deterministic tokenizer of Lemma 3, which keeps every observed hyperedge that contains the center and all of their 2-subsets that contain the center. It does not guarantee that a bounded, sampled view retains the subsets that separate the two hypergraphs. This scope is stated in Lemma 3 and at the start of App. B.3. Theorems 1 and 2 do not involve the tokenizer. Empirically, with the actual bounded tokenizer, the inclusion-aware encoder reaches 86.03 / 80.42 (homophilic / heterophilic), against 84.78 / 77.99 for a vanilla Transformer on the same tokens and 83.29 / 76.29 for mean pooling (Table 2(c)).

W2 and Q2. Pretraining control and missing-subset tokens

W2.1 and Q2. Pretraining control

"HGPM performs notably worse without pretraining, and the best objective is generic EMA reconstruction."

Q2: "Can the author design a pretraining control for a fair comparison?"

We provide a pretrained strong-baseline comparison (configuration 1) and a matched-input architecture–pretraining control (configuration 2).

Configuration 1: masked EMA pretraining adapted to conventional baselines

Node benchmarks (table below). We add masked EMA pretraining to HealHGNN, the strongest baseline in Table 1 (average rank 2.9). Its architecture, published per-dataset configuration and downstream training are unchanged. In pretraining, the content embeddings of 20% of nodes are replaced by a mask vector, an EMA teacher sees the unmasked input on the same hypergraph, and the student regresses the teacher's projected final-layer states at the masked nodes.

HealHGNN as reported and with masked EMA pretraining, and HGPM, on the node benchmarks; accuracy (%), mean ± s.d. The best value in each row is in bold.

DatasetHealHGNNHealHGNN + masked EMAHGPM (ours)
Citeseer75.1 ± 1.275.6 ± 1.377.1 ± 1.2
Pubmed88.8 ± 0.390.0 ± 0.489.7 ± 0.5
Cora-CA84.3 ± 1.784.4 ± 1.585.1 ± 0.2
DBLP-CA92.0 ± 0.392.1 ± 0.292.2 ± 0.2
Congress93.5 ± 0.894.2 ± 0.994.1 ± 1.0
Senate76.1 ± 4.176.1 ± 4.477.2 ± 4.8
Walmart68.2 ± 0.668.9 ± 0.671.2 ± 0.5
House77.2 ± 2.377.7 ± 2.379.2 ± 1.8
Homophilic mean85.185.586.0
Heterophilic mean78.879.280.4

The HealHGNN + masked EMA column is based on the official HealHGNN backbone with masked EMA pretraining added, using the same protocol as our method.

  • The pretrained baseline is at or above the reported one on every dataset: higher by 0.1–1.2 percentage points on seven of the eight datasets and equal on Senate. The homophilic / heterophilic means increase from 85.05 / 78.75 to 85.53 / 79.23.
  • With both models pretrained, HGPM has higher mean accuracy on six of the eight datasets, by 0.1–2.3 points. Its group means are 86.03 / 80.42, compared with 85.53 / 79.23 for HealHGNN + EMA. On Pubmed and Congress, HealHGNN + EMA is higher, by 0.3 and 0.1 points, respectively.

Drug benchmarks. An analogous control with KHGNN, the strongest baseline in Table 3, gives a similar picture. Adapted masked EMA pretraining improves KHGNN by 0.7 edge-classification F1 points on both HODDI and JADER (90.5 to 91.2 and 75.7 to 76.4), and HGPM remains 1.7 / 2.0 points above the pretrained KHGNN (control table in W3.1).

Configuration 2: identical tokens, with and without pretraining and structural biases

Encoder and pretraining on the node benchmarks, with identical subset tokens; group-mean accuracy (%), ten seeds.

EncoderPretrainingHomophilicHeterophilic
Vanilla TransformerNone83.175.9
Vanilla TransformerMasked EMA84.878.0
HGPMNone83.878.6
HGPMMasked EMA86.080.4

The first row is new; the others restate results in the paper (Table 2(c) and Table 1). The pretraining recipe is the same for the two encoders on every dataset (App. E.1).

Pretraining helps both encoders by a similar amount: 1.73 / 2.07 points for the vanilla Transformer and 2.28 / 1.84 for HGPM. The pretraining gain is therefore not specific to HGPM, in line with the remark that EMA reconstruction is generic. We use it as a standard component and do not claim it as a contribution; Table 2(e) shows only that it is the best of the four targets we tried. The structural biases add 0.70 / 2.66 points without pretraining and 1.25 / 2.43 with it.

W2.2. Missing-subset tokens

"Removing missing-subset tokens causes only a small drop, leaving the contribution unclear."

The "No negatives" row of Table 2(a) does not remove missing subsets. It removes only the perturbation-generated tokens; unobserved fragments of observed hyperedges remain, and in the full tokenizer these fragments make up 99.29–100% of sub tokens outside Congress (audit table in W1.1). Its 0.22 / 0.81 drop therefore measures one source of unobserved subsets, not their total contribution. The revised PDF relabels the row "No perturbation tokens".

Two results on the node benchmarks measure what the existence and source inputs contribute.

Evidence already in the paper (Table 2(d)). The composition bias bcompb^{\mathrm{comp}} is indexed by the roles (center, observed, unobserved) of the query and key tokens, which are derived from existence 1S\mathbf{1}_S and not from the source tag, so it is the channel through which attention distinguishes absent from present subsets. With the tokens and every other input unchanged, removing it lowers accuracy by 0.73 / 1.32 points (homophilic / heterophilic), and by 0.52–1.77 points on each of the eight datasets (Table 8, App. F).

New ablation (table below). We remove existence and source from the encoder's inputs: the existence embedding, the source embedding and bcompb^{\mathrm{comp}} are disabled in pretraining and fine-tuning, with tokens, inclusion edges and all other biases unchanged. This is the node-benchmark counterpart of the "HGPM without subset-state inputs" row of the drug-prediction control table in W3.1.

HGPM without existence and source inputs, on the node benchmarks; accuracy (%), mean over ten seeds. The first and last columns restate Table 8 (App. F) and Table 1.

DatasetObserved hyperedges onlyWithout existence and source inputs (new)HGPM
Citeseer73.374.777.1
Pubmed88.088.889.7
Cora-CA82.483.185.1
DBLP-CA90.090.692.2
Congress89.791.694.1
Senate71.674.377.2
Walmart65.466.171.2
House75.176.179.2
Homophilic mean83.484.386.0
Heterophilic mean75.577.080.4

Hiding these inputs lowers accuracy on all eight datasets, by 0.91–5.11 points, and by 1.76 / 3.42 points in the homophilic / heterophilic means. This is 8.0 / 4.2 times the drop of the "No perturbation tokens" row.

These results show that telling the encoder which subsets are observed, and how the unobserved ones were generated, helps node classification.

Reviewer_ELyvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a hypergraph model that explicitly represents both observed and unobserved node sets. It uses a Transformer to learn from these sets and their subset relationships. The method is evaluated on node classification and drug–side-effect prediction, with improvement on ten hypergraph benchmarks.

Critical strengths
  1. Explicitly representing unobserved subsets is an interesting way to capture patterns which I think previous hypergraph models have missed.
  2. The experiments cover ten datasets, report strong results, and examine the contributions of several model components.
Critical weaknesses
  1. From my understanding, the meaning of “existence” doesn't seem consistent: a subset absent from the data can receive label 1 or 0 depending on how it was sampled. This makes the interpretation of the learned patterns unclear.
  2. The two drug-prediction tasks can be more clearly distinguished. Additional controls are also needed to show whether the gains come from the proposed architecture or the subset-label information supplied to it.
Questions for authors

Please see weaknesses.

4 comments

W1. Consistency of existence

"From my understanding, the meaning of "existence" doesn't seem consistent: a subset absent from the data can receive label 1 or 0 depending on how it was sampled. This makes the interpretation of the learned patterns unclear."

Sec. 4.1 and App. C.1 give every sub token existence 1. Under that description, an unobserved fragment of an observed hyperedge would be presented as observed, and the same subset could have existence 1 in one view and 0 in another. We correct the description accordingly, as below and in the updated draft. The correction concerns the text only; the implementation and all reported results are unchanged.

In the implemented tokenizer, every non-center token receives the indicator of Sec. 3,

1S=1[S∈E],\mathbf{1}_S=\mathbf{1}[S\in\mathcal{E}],

where E\mathcal{E} is the set of hyperedges visible to the model. The rule is the same for every source. The source tag τ(S)∈{obs,sub,neg}\tau(S)\in\{\mathrm{obs},\mathrm{sub},\mathrm{neg}\} records how a token was generated and does not affect its existence. A subset absent from the visible hyperedge set therefore has existence 0, consistently across views.

We audited the implemented tokenizer's outputs on the eight node-classification benchmarks, covering 229,518 views, before pretraining masking (table below). The share of sub tokens with existence 0 is 99.29–100% outside Congress and 73.09% on Congress. The remaining fragments are themselves observed hyperedges. No membership errors or cross-view existence conflicts were found.

Existence of sub tokens in the implemented tokenizer's outputs. Counts are token occurrences across views; existence is checked against the visible hyperedge set. In the last column, a subset is identified by its node set and compared across the views in which it occurs within the same dataset.

Datasetsub tokensExistence 0Existence 1 (the fragment is itself a hyperedge)Share with existence 0Subsets with conflicting existence across views
Citeseer30,68930,52416599.46%0
Pubmed438,831435,7223,10999.29%0
Cora-CA77,64477,35628899.63%0
DBLP-CA1,762,5611,759,0133,54899.80%0
Congress31,00922,6638,34673.09%0
Senate66,56966,5690100.00%0
Walmart234,225233,65756899.76%0
House250,322250,3139100.00%0

For any retained adjacent pair with at least one observed endpoint, this rule gives the COMP/EMER/INHIB label defined in Sec. 3; (0,0)(0,0) pairs are outside these three regimes.

W2. The two drug tasks, and where the gain comes from

W2.1. The two drug-prediction tasks

"The two drug-prediction tasks can be more clearly distinguished."

The two tasks differ in what is predicted and in whether negative samples are used. In both, a record (G,s)(G,s) pairs a drug set GG with an adverse event (AE) ss.

  • Edge classification. The task is to predict which side-effect class an observed drug combination is reported with (1,710 classes on HODDI, 100 on JADER). It is scored per record (G,s)(G,s) with the label that the data assigns to it, so no negative samples are generated. We report F1 and AUROC.
  • Link prediction. The task is to tell observed records from negative samples: observed records are the positives, and sampled negative records are the negatives. We report AUROC and AUPRC.

What HGPM reads. For each subset SS of GG among its tokens, HGPM reads the state of (S,s)(S,s) in the training records: positive, negative, or unknown if no such record exists. We refer to these training-derived states as subset-state evidence, and to the channels that encode them as subset-state inputs. The token of GG itself has this input masked, and validation and test records are never read (App. C.3). This subset-state evidence is built from the same training records that every model is trained on, so HGPM uses no additional data. The conventional baselines do not take it as input.

W2.2. Where the gain comes from

"Additional controls are also needed to show whether the gains come from the proposed architecture or the subset-label information supplied to it."

Both structural encoding and the explicit state and source input channels contribute under the evaluated controls. The table below reports three separate comparisons on both datasets with ten seeds; these do not form an additive decomposition of the total gain. In edge-classification F1, HODDI / JADER:

  • Architecture. With inputs and pretraining matched, HGPM is 1.3 / 1.1 points above a vanilla Transformer.
  • Explicit state and source channels. With architecture and pretraining matched, the existence embedding, source embedding, and composition bias jointly add 1.2 / 1.6 points. Without them HGPM is still 0.5 / 0.4 above a pretrained KHGNN.
  • Pretraining. Adding adapted masked EMA pretraining to KHGNN improves it by 0.7 / 0.7, and HGPM remains 1.7 / 2.0 above it.

Drug-prediction controls (%), mean ± s.d. over the ten seeds of Table 3. The three control rows are defined below the table.

ModelHODDI, edge classification: F1HODDI, link prediction: AUROCJADER, edge classification: F1JADER, link prediction: AUROC
KHGNN90.5 ± 0.197.2 ± 0.175.7 ± 0.359.4 ± 0.1
KHGNN + masked EMA pretraining91.2 ± 0.297.4 ± 0.176.4 ± 0.360.3 ± 0.2
Vanilla Transformer + matched subset-state evidence91.6 ± 0.297.5 ± 0.177.3 ± 0.361.6 ± 0.2
HGPM without subset-state inputs91.7 ± 0.297.3 ± 0.176.8 ± 0.359.8 ± 0.3
HGPM (ours)92.9 ± 0.197.9 ± 0.078.4 ± 0.263.2 ± 0.1

The three control rows are defined as follows.

  • KHGNN + masked EMA pretraining. KHGNN is the strongest baseline in Table 3 (average rank 2.4). We mask the content embeddings of 20% of drug nodes and regress the final-layer states of the masked nodes onto an EMA teacher that sees the unmasked training hypergraph. The architecture and the downstream training are unchanged.
  • Vanilla Transformer + matched subset-state evidence. The same tokens, subset-state evidence, AE conditioning, target masking, pretraining objective and budgets as HGPM, without the structural attention biases.
  • HGPM without subset-state inputs. The existence embedding, source embedding, and composition bias are all disabled during pretraining (in both the student and teacher) and fine-tuning. Token sets, drug features, order, and the label-independent inclusion, direction, overlap and sibling inputs are kept. This ablation measures the joint contribution of the explicit state and source channels under unchanged supervision and retained token support.

Dear Reviewer,

As the discussion period is drawing to a close, we would like to check in on whether our response has resolved your concerns. We are happy to answer any further questions. If our response has addressed your concerns, we would be grateful if you would consider raising your rating. Thank you again for your time and thoughtful feedback.

Best Regards, Authors

Dear Reviewer,

With about 10 hours left in the discussion period, we would be happy to address any remaining questions or concerns you may have. Thank you again for your time.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.