ReCAST: Beyond Reconstruction—Learning Compositional Visual Responses

Shadow removal is usually determined by alignment with a reference shadow-free image. Under a fixed observation, however, distinct latent responses can yield similar appearance yet diverge once illumination or occlusion changes, so endpoint fidelity alone does not certify a reusable response. We introduce Missing-Transport Inversion (MTI), which recovers the radiance residual suppressed by occlusion so that it remains predictive under withheld illumination–occlusion recombinations, without assuming a uniquely identifiable physical decomposition. Based on this, we propose ReCAST, which grounds the occluded receiver with geometry-compatible non-shadow evidence, learns a shared compositional operator over receiver, illumination, and occlusion factors, and renders appearance with a response-conditioned diffusion decoder that also supports forward re-shadowing. On a controlled same-scene Transport Lattice, it predicts a fixed withheld pairing from complementary observations in a target-blind protocol, reducing normalized missing-transport error (NMTE) from $0.667$ to 0.589 against matched Cross-attention. On ISTD+, SRD, INS, and WSRD+, the same representation remains competitive in PSNR, showing that compositional response modeling does not trade away standard restoration quality. Code is available in the supplementary materials.