Natural language instructions encode various forms of how-to-knowledge: from traveling to a desired location, to preparing a dish for dinner, to executing biology lab experiments. Our research is motivated by the need for future AI agents executing and communicating through such natural language instructions. Procedural language, however, poses unique challenges, such as elided arguments in the instructions or assumed common sense knowledge about the world. Our research addresses aspects of these challenges. We focus on (1) interpreting recipes as fully specified action graphs and (2) composing new instructional recipes that can achieve a desired goal.

Code:
Recipe Interpretation with Action Graphs
Neural Checklist Model

Data:
Recipe Interpretation with Action Graphs
Now You're Cooking Dataset