Sofiia Druchyna will present her MSE Talk "Spectral Credit Reconstruction for GRPO in Long-Horizon RLHF" on Monday, April 20, 2026 at 10:30a in CS 401. Committee Members: Prof. Elad Hazan (advisor), Prof. Manoel Horta Ribeiro (reader). All are welcome to attend. Title: Spectral Credit Reconstruction for GRPO in Long-Horizon RLHF Abstract: Reinforcement Learning from Human Feedback (RLHF) relies on sequence-level rewards to train large language models, yet policy updates are typically applied uniformly across all tokens in a generated completion. This uniform credit assignment ignores the temporal structure of language generation and contributes to high-variance gradient estimates in long-horizon settings. We propose a spectral credit assignment method for Group-Relative Policy Optimization (GRPO) that redistributes scalar advantages across token positions using a low-dimensional spectral parameterization. Our approach represents token-level credit as a smooth combination of fixed low-frequency basis functions, with mixture weights predicted by a state-dependent gating network operating on pooled hidden states. This construction preserves the expected GRPO update while constraining token-level variation to a controlled, low-frequency subspace, thereby reducing variance without introducing additional critics or token-level rewards. We show that our method interpolates smoothly between standard GRPO and structured token-level credit assignment, offering a simple and stable mechanism for incorporating temporal structure into RLHF training.