rt-hierarchy.github.io - RT-H: Action Hierarchies Using Language

home > domains > rt-hierarchy.github.io

Description: RT-H: Action Hierarchies Using Language

Example domain paragraphs

Language provides a way to break down complex concepts into digestible pieces. Recent works in robot imitation learning have proposed learning language-conditioned policies that predict actions given visual observations and the high-level task specified in language. These methods leverage the structure of natural language to share data between semantically similar tasks (e.g., "pick coke can" and "pick an apple") in multi-task datasets. However, as tasks become more semantically diverse (e.g., "pick coke ca

Your browser does not support the video tag.

Language-conditioned policies in robotics leverage the structure of natural language to share data between semantically similar tasks (e.g., "pick coke can" and "pick an apple") in multi-task datasets. But as tasks become more semantically diverse (e.g., "pick the apple" and "knock the bottle over" below), sharing data between tasks is much harder.

Links to rt-hierarchy.github.io (5)

• tedxiao.me Ted Xiao - Google DeepMind
• jonathantompson.github.io Jonathan Tompson
• debidatta.github.io Debidatta Dwibedi
• txizzle.github.io Ted Xiao - Google DeepMind
• jonathantompson.com Jonathan Tompson