Research highlights

• Skill induction and planning with latent language.
  

  Pratyusha Sharma, Antonio Torralba and Jacob Andreas. ACL 2022.

  

  How can we train agents to plan & act in complex environments? We build models that “plan out loud”, using language as both a supervisory signal and internal representation. These agents improve significantly over standard imitation learning approaches, while producing internal representations of their long-range plans and generalizing to new goals using knowledge learned from text.

• Implicit representations of meaning in neural language models.
  

  Belinda Z. Li, Maxwell Nye and Jacob Andreas. ACL 2021.

  

  Language models (trained to predict missing words in sentences and paragraphs) learn to build structured representations of the state of the world. These representations support model-internal reasoning about the consequences of actions (like mixing a beaker or unlocking a door with a key). The organization of semantic information in language model embeddings bears striking resemblance to formal meaning representations like DRT and file change semantics from the linguistics literature, suggesting that it's possible to discover a little bit about how meaning works with nothing but text as training data.

• Compositional explanations of neurons.

  Jesse Mu and Jacob Andreas. NeurIPS 2020.

  

  We explain the behavior of deep network models by approximating each neuron with an executable program or logical form. These compositional explanations let us automatically quantify (aspects of) the complexity, interpretability, and naturalness of learned abstractions in vision and language models. They surface a number of surprising successes (image classifiers discover abstract functional categories like "sports facility" without supervision) but also sources of brittleness (vision models easily confuse washing machines with viaducts and cribs with fire escapes, and language processing models are easily tricked by surprising combinations of pronouns).

Full paper list

2022

• Tracing Knowledge in Language Models Back to the Training Data.

  Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, Kelvin Guu. Preprint.

• Compositionality as Lexical Symmetry.

  Ekin Akyürek, Jacob Andreas. Preprint.

• Correcting Robot Plans with Natural Language Feedback.

  Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Torralba, Jacob Andreas, Dieter Fox. RSS 2022.

• Quantifying Adaptability in Pre-trained Language Models with 500 Tasks.

  Belinda Z. Li, Jane Yu, Madian Khabsa, Luke Zettlemoyer, Alon Halevy, Jacob Andreas. NAACL 2022.

• Skill induction and planning with latent language.

  Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. ACL 2022.

• Natural Language Descriptions of Deep Visual Features.

  Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba and Jacob Andreas. ICLR 2022 (oral).

  Press: MIT News.

• Subspace Regularizers for Few-Shot Class Incremental Learning.

  Afra Feyza Akyuürek, Ekin Akyürek, Derry Wijaya and Jacob Andreas. ICLR 2022.

2021

• How do neural sequence models generalize? Local and global context cues for out-of-distribution prediction.

  Anthony Bau and Jacob Andreas. EMNLP 2021.

• The low-dimensional linear geometry of contextualized word representations.

  Evan Hernandez and Jacob Andreas. CoNLL 2021.

• Leveraging language to learn program abstractions and search heuristics.

  Catherine Wong, Kevin Ellis, Joshua B. Tenenbaum and Jacob Andreas. ICML 2021.

• Implicit representations of meaning in neural language models.

  Belinda Z. Li, Maxwell Nye and Jacob Andreas. ACL 2021.

• What context features can transformer language models use?

  Joe O’Connor and Jacob Andreas. ACL 2021.

• Lexicon learning for few-shot sequence modeling.

  Ekin Akyürek and Jacob Andreas. ACL 2021.

• Multitasking inhibits semantic drift.

  Athul Paul Jacob, Mike Lewis and Jacob Andreas. NAACL 2021.

• Representing partial programs with blended abstract semantics.

  Maxwell Nye, Yewen Pu, Matthew Bowers, Jacob Andreas, Joshua B. Tenenbaum, Armando Solar-Lezama. ICLR 2021.

• Learning to recombine and resample data for compositional generalization.

  Ekin Akyürek, Afra Feyza Akyürek and Jacob Andreas. ICLR 2021.

2020

• Compositional explanations of neurons.

  Jesse Mu and Jacob Andreas. NeurIPS 2020 (oral).

• A benchmark for systematic generalization in grounded language understanding.

  Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt and Brenden Lake. NeurIPS 2020.

• Good-Enough Compositional Data Augmentation.

  Jacob Andreas. ACL 2020.

• Unnatural language processing: bridging the gap between synthetic and natural language data.

  Alana Marzoev, Sam Madden, Frans Kaashoek, Mike Cafarella and Jacob Andreas. NeurIPS workshop on Emergent Communication.

2019

• A survey of reinforcement learning informed by natural language.

  Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson and Tim Rocktäschel. IJCAI 2019.

• Measuring compositionality in representation learning.

  Jacob Andreas. ICLR 2019.