What Does Text Add to Forecasting? A Controlled Audit

In our controlled studies, text-induced forecast changes do not consistently translate into lower error, and embedding-source rankings depend on the evaluation metric. We distinguish forecast response, control-relative benefit, semantic selectivity, and embedding-source effects using paired event worlds and matched training. Across 240 synthetic processes, a fixed Qwen instruction model favors correct over incorrect event descriptions on average and moves forecasts in the event's direction, but the benefit is concentrated in level changes and reverses for capacity constraints. Follow-up comparisons on the same inspected cohort have negative mean scores against no auxiliary context ($-0.1283$) and metadata only ($-0.1579$): resampling intervals are negative, while conservative bounded intervals include zero. Across 16 matched GPT-2 training units, original and permuted pretrained embeddings both reduce an integrated unsigned loss gap relative to random embeddings, yet final error and signed-benefit readouts rank the sources differently. These findings identify concrete dependence on the comparator, event family, and readout, without establishing general semantic benefit or a shared semantic mechanism.