[Call for participation] Interactive Grounded Language Understanding in a Collaborative Environment (IGLU) Competition@NeurIPS2021

Research
1

Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research in this direction, we propose the NeurIPS IGLU competition: Interactive Grounded Language Understanding in a Collaborative Environment.

The primary goal of the IGLU competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. Understanding the complexity of the challenge, we split it into sub-tasks to make it feasible for participants.

This research challenge is naturally related, but not limited, to two fields of study: Natural Language Understanding and Generation (NLU/G) and Reinforcement Learning (RL). Therefore, the suggested challenge can bring two communities together to approach one of the important challenges in AI. Another important aspect of the challenge is the dedication to perform a human-in-the-loop evaluation as a final evaluation for the agents developed by contestants.

The goal of our competition is to approach the following scientific challenge: How to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment? By the interactive agent we mean that the agent is able to follow the instructions correctly, is able to ask for clarification when needed, and is able to quickly adapt newly acquired skills, just like humans are able to do while collaboratively interacting with each other.

Tasks and Application Scenarios:
Given the current state of the field, our main research challenge might be too complex to suggest a reasonable end-to-end solution. Therefore, we split the problem into the following concrete research tasks:

Architect Task: Given target structure, generate step instructions for the Builder

Builder Task: Given Architect-Builder conversation, build target structure:

Prizes:

Architect Task:

1st place - 5K $

2nd place - 1.5k $

3rd place - 500 $

Builder Task:

1st place - 5K $

2nd place - 1.5k $

3rd place - 500 $

Timeline:
July 26 – Stage 1 begins;
(Tentative) October 15 – Stage 1 ends;
October 22 – Stage 2 begins by deploying the top-3 performing agents for human evaluation;
November 26 – The results of Stage 2 are posted, and the list of winning teams per task is released;
December 6 – NeurIPS 2021 begins.

Upcoming workshops:

To make it even easier for you to onboard, we will arrange workshops per task:

During the workshops, our team will walk you through setup, available baselines, and training environment (for Builder task). You will have a great opportunity to ask any questions, which we probably can answer :slight_smile:

Guest Lectures:

If you have missed our guest lectures. Here are the links to recordings:

For more frequent updates:

2 Likes