BAO-Net: Bayesian Agentic Grounding for Reliable Multimodal Reasoning

Multimodal grounding in frozen language models is often unstable during modality injection, making it difficult to attribute specific improvements to a particular mechanism rather than to incidental optimization dynamics. This work evaluates IGCA, a zero-gated cross-attention bridge designed to preserve pre-trained behavior initially while facilitating gradual multimodal adaptation. Upon integrating IGCA across six decoder layers of the Falcon 40B, open_llama_30b and Bloom-7b1 backbone, the model utilizes 40,404,961,288 trainable adapter parameters for training purpose only. Verification of the zero-gate mechanism confirms precision: the maximum output difference between the “gate-off” and “gate-initialized” models is 0.00e+00. This demonstrates that the bridge acts as a no-op initially, establishing a stable starting point for effective multimodal adaptation. Training over 59,500 steps reduces the loss from 3.9403 to 0.0007, with the final ten loss values recorded as: [0.0001, 0.0003, 0.0005, 0.0002, 0.0005, 0.0011, 0.0005, 0.0006, 0.0005, 0.0007]. The best “held-out” gate-on checkpoint achieves a loss of 2.2201, compared to 4.3469 for the gate-off baseline, a reduction of 2.1268 nats (48.9%). The average gate-off loss is 4.3423, while the average gate-on loss is 2.2498, representing an improvement of 1.9075 nats. At the best checkpoint, the gate remains slightly open, with an average |tanh(α)| = 0.0713. These results indicate that zero-gated multimodal adaptation improves grounding without destabilizing the frozen language model.