jumpstart-rl.github.io - Jump-Start RL

Example domain paragraphs

Reinforcement learning (RL) provides a theoretical framework for continuously improving an agent's behavior via trial and error. However, efficiently learning policies from scratch can be very difficult, particularly for tasks with exploration challenges. In such settings, it might be desirable to initialize RL with an existing policy, offline data, or demonstrations. However, naively performing such initialization in RL often works poorly, especially for value-based methods. In this paper, we present a met

We’re introducing a meta-algorithm called Jump-Start Reinforcement Learning (JSRL) that can use a pre-existing policy of any form to initialize any type of RL algorithm. JSRL uses two policies to learn tasks: a guide-policy, and an exploration-policy. The exploration-policy is an RL policy that is trained online with new experience, and the guide-policy is a fixed, pre-existing policy of any form. In this work, we focus on scenarios where the guide-policy is learned from demonstrations, but many other kinds

At the beginning of training, we roll out the guide-policy for a fixed number of steps so that the agent is closer to goal states. The exploration-policy then takes over and continues acting in the environment to reach these goals. As the performance of the exploration-policy improves, we gradually reduce the number of steps that the guide-policy takes, until the exploration-policy takes over completely. This process creates a curriculum of starting states for the exploration-policy such that in each curric

Links to jumpstart-rl.github.io (3)

tedxiao.me Ted Xiao - Google DeepMind
karolhausman.github.io Karol Hausman
txizzle.github.io Ted Xiao - Google DeepMind