Colosseum
Description: NEWTON: Are Language Models Capable of Physical Reasoning?
Training language models on extensive unprocessed text data has yielded impressive advancements in natural language processing (NLP), particularly in tasks such as question answering and reading comprehension. These models, through their contextualized representations, have been proven in numerous studies to encapsulate syntactic, semantic, word sense, and common-sense knowledge. However, there has been limited exploration of their physical reasoning abilities, specifically concerning the crucial attributes
Through extensive empirical analysis, our results highlight the capabilities of LLMs for physical reasoning. Furthermore, the NEWTON platform demonstrates its potential for evaluating and enhancing language models, paving the way for their integration into physically grounded settings.
NEWTON Repository, includes the identification and shortlist of objects and attributes, and obtaining a set of consistent object-attribute annotations.
Colosseum