TAFP-DNA: TEMPERATURE-ANNEALED FOCUSED POOLING FOR DNA FOUNDATION MODELS

Fixed pooling strategies in current DNA foundation models poorly adapt to diverse Transcription Factor (TF) binding patterns and can make predictions vulnerable to sequence-level statistical biases such as Guanine-Cytosine (GC) content. To tackle these issues, we propose Temperature-Annealed Focused Pooling (TAFP), an attention pooling head with a temperature-scaled softmax. Inspired by simulated annealing, TAFP gradually lowers the temperature during fine-tuning to sharpen the attention distribution, moving from broad global aggregation to a focus on informative local regions; at inference, the temperature is kept fixed. When a known motif is available, an optional weak supervision loss aligns attention with motif sites. By exploiting the Reverse-Complement (RC) symmetry of DNA, RC training augmentation combined with averaging of forward- and RC-strand predictions yields RC-invariant predictions without an extra RC consistency loss. On ten TF binding prediction tasks from the Genome Understanding Evaluation (GUE) benchmark, the resulting TAFP-DNA achieves a state-of-the-art average Matthews Correlation Coefficient (MCC) of 0.7381 over seven baselines, and under the same backbone, focused pooling outperforms CLS and mean pooling by 0.044 and 0.034 MCC. With motif supervision, the motif recovered from attention weights matches the known MAF profile. TAFP is a lightweight module that can be added to pretrained DNA models without architectural changes.