Read-Write Reinforcement Learning: Data-Centric Experience Correction for Robust Reinforcement Learning Under Uncertainty

Abstract

While Reinforcement Learning (RL) excels in solving complex sequential decision-making tasks, its performance often degrades catastrophically in real-world environments plagued with sensory noise and uncertainty. Existing robust RL methodologies are overwhelmingly model-centric, focusing on hardening the policy to whist stand these perturbations rather than addressing the quality of the underlying data. To bridge this gap, we introduced Read-Write Reinforcement Learning (RW-RL), a novel data-centric paradigm that challenges the traditional assumption of a static experience replay buffer. Throughout extensive empirical evaluations across multiple environments subjected to varying levels of observation noise, we demonstrate that RW-RL is capable of modifying stored transitions without destabilizing or collapsing training, and furthermore, our results establish that this methodology yields policies that are on par with established policy-centric methods, and in severe noise scenarios, able to surpass them. Ultimately, RW-RL proves that actively rectifying data quality provides a viable pathway to building robust, self-improving systems in uncertain environments.