SAPO: Step-Aligned Policy Optimization for Reasoning-Based Generative Recommendation

Generative recommendation treats next-item prediction as autoregressive item-identifier generation. Specifically, items are encoded as semantic identifiers (SIDs), which are short coarse-to-fine token sequences whose early tokens capture broad semantics and later tokens refine them. Recent work augments this paradigm with reasoning traces and optimizes them via reinforcement learning with verifiable rewards, typically an outcome-reward algorithm scored on the generated SID. However, in realistic recommendation, such reward design becomes a single scalar per rollout; when a generated SID mismatches, it cannot identify which SID-token prediction caused the mismatch and may penalize matched SID-token positions together with the mismatched position. We identify that the natural unit of credit assignment in this setting is a single reasoning step (one thinking block paired with one SID token). We instantiate this idea in SAPO (Step-Aligned Policy Optimization): rather than broadcasting one advantage to the whole response, SAPO computes a separate group-relative advantage for each reasoning step and applies it only to the corresponding thinking block and SID token. Across three real-world recommendation datasets, SAPO stabilizes RL training and achieves competitive or improved performance, with the largest gains where fine-grained credit assignment matters most. Our results suggest that reinforcement-learning objectives for structured generation should mirror the decoder's own decomposition of the output. Our code is available at https://anonymous.4open.science/r/SAPO.