acceptodds
Under review as a conference paper at ICLR 2027

VTPE: Virtual Trajectory Position Encoding for Biomolecular Complex Structure Prediction

Abstract

AlphaFold-style co-folding methods have achieved near-experimental accuracy on protein monomers, yet structure prediction for biomolecular complexes remains challenging. A key difficulty is that biomolecular complexes contain many atoms that engage in long-range interactions: the position of one atom constrains the positions of numerous other atoms at varying distances, all at long range. Diffusion-based structure prediction resolves these constraints progressively over denoising steps, but supervision is defined only on the final denoised structure, leaving the relationships among intermediate states unmodeled. Consequently, the atomic adjustment process and the coordination among distant atoms remain difficult to capture. In this paper, we view these intermediate structures as a virtual trajectory and propose Virtual Trajectory Position Encoding (VTPE), which encodes atomic coordinates along this virtual trajectory as positional encoding. This enables the model to represent how distant atoms move relative to one another over the course of denoising. Since long-range interactions are precisely the couplings that produce such coordinated adjustments, exposing the model to these adjustments provides a direct signal for learning long-range interactions. We evaluate VTPE at multiple parameter scales of Protenix, the open-source reproduction of AlphaFold 3 (AF3), and observe consistent improvements over the corresponding baselines across benchmarks with limited additional overhead. Experiments confirm the contribution of virtual trajectory position encoding to complex structure prediction.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Complex Structure Prediction, Artificial Intelligence, Position Encoding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1CY8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a position-encoding module for biomolecular structure prediction. Hidden atomic representations are linearly projected to three-dimensional virtual coordinates, which control rotary transformations of queries and keys in subsequent attention layers. The three axes use interleaved, balanced frequency schedules, with learnable frequency scaling. The paper proves relative-position consistency, norm preservation, local isotropy of the positional operator's perturbation response, and a fixed-content logit sensitivity bound. Experiments at three Protenix scales report improvements on most metrics across PoseBusters, RecentPDB, and selected PXM-Legacy subsets.

Critical strengths
  1. The module is conceptually compact. Using learned coordinates to modulate attention is an interesting way to inject structured relative information
  2. The empirical signal is promising across several settings. At 20,000 training steps, Protenix-base improves PoseBusters LDDT-mean from 63.61 to 67.34 and RMSD-distance from 11.58 to 9.82.
  3. Improvements also appear in aggregate RecentPDB and PXM-Legacy results and at smaller model scales.
Critical weaknesses
  1. The virtual-trajectory interpretation is stronger than the construction establishes. Definition 1 produces unsupervised three-dimensional projections of hidden features across attention layers. These are not shown to be physical atomic coordinates or the coordinate states of successive diffusion steps. The introduction motivates the method through denoising trajectories, while the formal definition is layerwise; the relationship between these two indices needs to be explicit. In summary, the current coordinate/trajectory is the map from the feature, which the author interprets as relative motion in the diffusion steps. The evidence linking diffusion step motion and trajector is missing. The current evidence supports more like "feature-related positional encoding is more effective. "
  2. Local isotropy is not full three-dimensional rotational symmetry. Proposition 2 concerns the leading quadratic term of a Frobenius-norm response to small positional perturbations. It does not establish rotational invariance of content-weighted attention logits, equivariance of the coordinate projection, or equivariance of the complete model. Even for one equal-frequency group, the exact response is proportional to the sum over axes of 1 - cos(theta * delta_axis), which is not generally a function only of the displacement norm at finite displacement. The theorem itself is appropriately local, but the broader “3D symmetry-preserving” language and Figure 3 caption need qualification. A transformation law for learned coordinates and controlled global-rotation tests would address the practical claim.
  3. The controls do not establish that dynamic geometry explains the improvements. Table 4 removes frequency scaling or changes axis allocation, but does not compare dynamic coordinates with fixed learned positions, random/frozen projections, current noisy coordinates, or a parameter-matched additive geometric encoding.
  4. Evaluation and implementation details are insufficient for interpreting the reported magnitudes. “RMSD-mean” is an average of success rates at 5, 10, 15, and 20 Å, not a mean distance. This is not a common metric used by other complex structure prediction works, as they usually use mean RMSD or dockq success rate.
Questions for authors

Which attention modules produce and consume the virtual coordinates, and how do layer indices, diffusion steps, and the virtual steps in Figure 2 correspond?

Reviewer_jWHUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper adds a linear head at each attention layer of Protenix that maps hidden states to 3D “virtual coordinates,” then uses these as 3D RoPE positions in the next layer. The coordinates get no supervision. Protenix base/mini/tiny are trained with and without VTPE on a 10k WeightedPDB subset (20k steps, crop 128), and VTPE improves LDDT and RMSD success rates on PoseBusters, RecentPDB and PXM-Legacy.

Critical strengths
  1. The comparison is fair. Baseline and VTPE share the same recipe and compute, and the gains show up across three model sizes and most subsets. Some gains are large. Protenix-mini improved from 45.98 to 55.36 LDDT on PoseBusters, which is hard to explain away as noise.
  2. The idea is simple and effective. Letting the network produce its own 3D positions and feeding them back into RoPE needs only one linear head per layer. It can be added to any AF3-style model without changing the loss or the data pipeline, so it would be easy to adopt.
Critical weaknesses
  1. All experiments run in a heavily undertrained regime. Protenix-base gets only 36% of PoseBusters ligands under 5 Å, while released AF3 and Protenix are around 75 to 80% under 2 Å. The gap also shrinks with training: the base model gains 12.7 LDDT at 10k steps but only 3.7 at 20k, and tiny is basically tied at 20k. This looks more like faster early convergence than a better model.
  2. The long-range claim is not supported. On RecentPDB, intra_prot improves about as much as prot_prot and ligand_prot, and no interface metric like DockQ or iLDDT is reported. Table 3 also seems to evaluate each crop size on a different and much smaller subset. Its percentages are multiples of 1/22 at crop 256 and 1/89 at crop 384, while the default row uses 210 complexes.
  3. The symmetry analysis does not carry over to learned coordinates. Since the coordinates come from a free linear map, the model can rescale each axis by itself, so the anisotropy Prop 2 removes can be learned away anyway. The gaps in Table 4 are more likely from initialization or noise than from symmetry.
Questions for authors
  1. Does “initialized from Protenix” mean the released weights or random initialization?
  2. Does VTPE still help when fine-tuning from the released checkpoint?
  3. Where exactly is VTPE inserted? If it sits in the atom transformer, that attention is local, so it cannot capture long-range interactions. What is a virtual step? The abstract describes the trajectory as denoising intermediates, Section 3 defines it over layers, and the x-axis in Figure 2 goes to 1200.
  4. Why not use the actual noisy coordinates or the current denoised estimate as RoPE positions? That seems like the most natural baseline.
  5. How is this different from IPA in AlphaFold2, which also projects hidden states to 3D points and uses their distances inside attention?
1 comment

I thank the authors for providing a detailed response that addresses my main concerns. I will therefore maintain my positive score.

Reviewer_3ccCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel positional encoding for AF3-like structure prediction model, VTPE, that can encode the atomic coordinates along a virtual trajectory of the intermediate states during the diffusion process. Specifically, VTPE projects intermediate atom representations into virtual 3D coordinates and uses them to construct layer-dependent rotary encodings for attention queries and keys. Interleaved, axis-balanced frequency allocation and learnable frequency scaling adapt the encoding to three coordinate components. The method adds no intermediate-coordinate supervision and retains the backbone training losses. Evaluations on Protenix-base,-mini, and -tiny report improvements in most settings across PoseBusters, RecentPDB, and selected PXM-Legacy subsets.

Critical strengths
  1. A simple, lightweight architectural design without additional supervision. A lightweight linear coordinate projection and rotary encoding reuse intermediate representations while retaining the original training objective. The modular design offers a potentially practical way to modify attention without intermediate-structure labels and may be reusable beyond the tested backbone.

  2. Compelling results across model sizes and benchmarks. The experiments report gains in most evaluated settings, especially for base and mini, under the matched training budgets described in Sec. 4.1. The results support the motivation of the positional encoding modification.

Critical weaknesses
  1. The geometry-specific contribution and long-range mechanism are not isolated. The virtual coordinates receive no direct geometric supervision (Definition 1). It remains unclear whether they capture meaningful molecular geometry or serve as an unconstrained latent parameterization of attention. Table 4 ablates frequency choices rather than isolating this distinction. A key missing control is an attention-bias branch generated directly from hidden states, without virtual coordinates or RoPE, with approximately matched additional parameters and compute, inserted into the same modules with identical masks. This would distinguish the proposed encoding from more general input-dependent attention modulation.

    The long-range explanation also lacks architectural detail. Original AF3 uses sequence-local atom attention and global token attention (AF3 Supplement, Sec. 3.2; Algorithms 7 and 20). In contrast, VTPE Eq. 6 sums over all atoms without showing a mask, while Sec. 4.1 does not identify the modified modules or window settings. If the original hard masks are retained, RoPE cannot directly connect blocked atom pairs; improvements could nevertheless propagate indirectly through global token attention. Crop-size experiments (Sec. 4.3) do not resolve this distinction.

  2. The trajectory and symmetry interpretations go beyond the demonstrated properties. Figure 2 contains approximately 1,200 virtual steps, whereas Definition 1 describes attention-layer states. Their correspondence to specific blocks, repeated calls, and diffusion steps is unspecified. Moreover, Eq. 9 measures changes in average protein–ligand centroid separation, not stabilization of full atomic configurations. Stable centroid separation does not imply stable individual coordinates or relative geometry. A layer-resolved, per-sample analysis is needed to establish what stabilizes and its relationship to structural predictions (Fig. 2; Sec. 3.3).

    The theoretical scope also needs qualification. Propositions 1–3 establish relative-position consistency and norm preservation, leading-order local isotropy of the positional operator, and a coordinate-perturbation bound with content features fixed. These do not establish full SO(3) equivariance, layer-wise convergence, or physical meaning of the virtual coordinates (Apps. B.2–B.4). In particular, local isotropy should not be conflated with arbitrary-rotation equivariance; the stated fixed-axis encoding and coordinate projection provide no such architectural guarantee (Definitions 1–2).

Questions for authors
  1. Primary control: Can you provide the parameter-matched hidden-state attention-bias baseline described in W1, preserving insertion locations, masks, and training conditions?
  2. Primary trajectory analysis: How are the 1,200 steps constructed? Please map them to blocks, repeated calls, and diffusion steps, and supplement centroid separation with per-sample changes in interatomic distances or pairwise distance matrices.
  3. Geometric supervision — secondary question: Why are the virtual coordinates left without explicit geometric supervision (Definition 1)? Have auxiliary relative-distance or contact-geometry losses been explored? A comparison of geometrically constrained and unconstrained coordinates could test whether imposing structural relationships helps prediction. The absence of such a loss is not itself a weakness; this is a diagnostic suggestion, not a requirement to force every layer to reproduce the final structure.
  4. Architecture and masks: Which atom/token modules receive VTPE? Are windows and masks unchanged? Please reconcile Eq. 6 with the implementation; module-specific ablations would help identify the route of any long-range benefit.
  5. Theory clarification: Proposition 3 defines unrotated Q/K but omits the relative rotary operator from its logit, whereas App. B.4 includes it. Please reconcile these definitions. Are learnable frequency scales shared across axes? Also, since Definition 2 already shares each frequency across axes, please distinguish the role of u = 6 in frequency coverage from the frequency-balance condition in Proposition 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.