Abstract: 

In recent years, model-free reinforcement learning has demonstrated its significant success in many fields. However, these algorithms suffer from high sample complexity and require a massive number of interactions with the environment, which can be expensive in the physical world. 

Model-based reinforcement learning is considered to be a promising approach to reduce sample complexity. However, the theoretical understanding of such methods has been rather limited. In this talk, I'll present an algorithmic framework for designing and analyzing model-based RL algorithms with theoretical guarantees. We design a meta-algorithm with a theoretical guarantee of monotone improvement to a local maximum of the expected reward. The meta-algorithm iteratively builds a lower bound of the expected reward based on the estimated dynamical model and sample trajectories and then maximizes the lower bound jointly over the policy and the model. Instantiating our framework with simplification gives a variant of model-based RL algorithms Stochastic Lower Bounds Optimization. Experiments demonstrate that our algorithm achieves state-of-the-art performance when only one million or fewer samples are permitted on a range of continuous control benchmark tasks.