- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the interaction between token and contextual information in natural language inference and proposes IC³ (Input-Conditioned Complex Convolution), an attention-free method. Building on existing complex-valued token–context representations, the method generates a complex value sequence and a dynamic convolution kernel from the input, performs convolutional propagation using zero-padded Fourier transforms, and uses the real part of the convolution output to update token representations. The theoretical analysis characterizes the structure and affine dimension of the conditional convolution operators, their approximation error using fixed operators, and the effects of changes in the values and operators on the output. Experiments on ANLI, FOLIO, and a reduced-data SNLI setting compare models of similar size trained from scratch and show that IC³ achieves higher mean accuracy than the selected baselines. Further operator comparisons, ablations, and response interventions indicate that input-conditioned propagation and the imaginary response contribute to prediction.
Critical strengths
- IC³ achieves higher mean accuracy than the selected baseline models of comparable size across five evaluation settings on ANLI, FOLIO, and reduced-data SNLI. The authors assess the stability of the results using ten random seeds and paired confidence intervals. Comparisons with alternative propagation operators and component ablations further demonstrate the contributions of input-conditioned propagation, complex-valued representations, and the imaginary response to model performance.
- The paper provides a detailed mathematical analysis of the conditional propagation operators in IC³, including their structure, affine dimension, fixed-operator approximation error, and the decomposition of output changes into value and operator variation. Response-spectrum analysis, low-dimensional projections, and interventions such as response swapping further examine how the input-conditioned operators vary and how this variation affects prediction.
Critical weaknesses
- IC³ directly inherits the Complex–Vector Token Representations proposed by Zhang and Sheng [1] and replaces the original Token–Context Attention with input-conditioned complex convolution. However, the only direct comparison with Token–Context Attention appears in the ANLI-R2 ablation in Table 4. In contrast, Table 3 compares other propagation operators on ANLI-R1, R2, and R3. Given that the main contribution is a new propagation mechanism, the lack of a systematic comparison with the most directly related prior method across multiple evaluation settings leaves insufficient evidence that the new operator offers a consistent performance advantage over the original method.
- Although the authors analyze the theoretical properties and internal mechanisms of IC³ in considerable detail, the experiments primarily focus on classification accuracy and the effects of the operators on prediction. In contrast, the directly preceding Token–Context Attention work also evaluates parameter efficiency, computational cost, adversarial robustness, and cross-dataset transfer. This paper does not establish whether these properties are preserved or improved after replacing the propagation operator. In particular, given the use of FFT-based convolution and the emphasis on computational efficiency, the absence of direct comparisons of runtime, FLOPs, and GPU memory usage leaves the tradeoff between performance and computational cost relative to the original method unclear.
Questions for authors
- Could the authors explain why the direct comparison with Token–Context Attention is limited to ANLI-R2 and provide further evidence that the performance advantage of IC³ extends to the other main evaluation settings? Please also clarify whether architectural factors other than the propagation operator were strictly controlled in this comparison.
- The paper implements input-conditioned complex convolution using FFTs and analyzes the theoretical complexity of the propagation stage, but does not compare actual computational costs with the original Token–Context Attention method. Could the authors report runtime, FLOPs, and GPU memory usage to help assess the tradeoff between performance and computational cost?
- The experiments in Section 5.1 show that replacing the dynamic response in Layer 0 with the mean response reduces accuracy by approximately 9.58 percentage points, whereas analogous interventions applied individually to Layers 1–11 produce absolute mean accuracy changes of no more than 0.06 percentage points. Meanwhile, Figure 2(a) shows substantial structure in response variation in later layers. How do the authors explain this pronounced difference across layers? Does it suggest that the predictive contribution of input conditioning is concentrated in the first layer? Do the existing experiments adequately support the need for dynamic input-conditioned responses in all 12 layers?