piqtopt.github.io - PI-QT-Opt

Example domain paragraphs

The predictive information , the mutual information between the past and future, has been shown to be a useful representation learning auxiliary loss for training reinforcement learning agents, as the ability to model what will happen next is critical to success on many control tasks. While existing studies are largely restricted to training specialist agents on single-task settings in simulation, in this work, we study modeling the predictive information for robotic agents and its importance for general-pu

PI-QT-Opt combines a predictive information auxiliary similar to that introduced in PI-SAC with the QT-Opt architecture. We define the past (X) to be the current state and action, (s, a), and the future (Y) to be the next state, next optimal action, and reward, (s_0, a0, r). A state s includes an RGB image observation and proprioceptive information. Image observations are processed by a simple conv net, the output of which is mixed with action, proprioceptive state, and the current task context using additi

We find that adding a predictive information auxiliary loss is an easy way to give substantial performance improvements to our chosen RL algorithm, as in Lee et al. which introduced Predictive Information Soft Actor-Critic (PI-SAC). However, we note that PI-SAC on its own was unable to solve our tasks, yielding close-to-zero success rates, and neither was SAC, which may indicate that the choice of base RL algorithm is still critical.

Links to piqtopt.github.io (3)

tedxiao.me Ted Xiao - Google DeepMind
kuanghuei.github.io Kuang-Huei Lee | Personal Website
txizzle.github.io Ted Xiao - Google DeepMind