Dialogue systems and conversational agents – including chatbots, personal assistants and voice control interfaces – are becoming increasingly widespread in our daily lives. NIPS is sponsoring an open competition to create a chatbot that can hold an intelligent conversation with a human partner.
|Register as a team member||Register as a human evaluator volunteer|
|Participant Registration||Volunteer Registration|
Recent advances in machine learning have sparked a renewed interest for dialogue systems in the research community. In addition to the growing real-world applications, the ability to converse is closely related to the overall goal of AI. This NIPS Live Competition aims to unify the community around the challenging task: building systems capable of intelligent conversations. Teams are expected to submit dialogue systems able to carry out intelligent and natural conversations about specific news articles with humans. At the final stage of the competition participants, as well as volunteers, will be randomly matched with a bot or a human to chat and evaluate answers of a peer. We expect the competition to have two major outcomes: (1) a measure of quality of state-of-the-art dialogue systems, and (2) an open-source dataset collected from evaluated dialogues.
|April, 2017||Registration is open. Registered teams are allowed to submit applications and solutions for the Qualification Round.|
|18th of June, 2017||Qualification Round is closed.|
|25th of June, 2017||Notification of acceptance. Teams for the Human Evaluation Round are selected.|
|24th-30th of July, 2017||Human Evaluation Round. One week NLP summer school. Teams, school participants and volunteers evaluate solutions of qualified teams. Evaluation is done remotely via a messaging platform.|
|1st of September, 2017||1st Dataset is published. Data collected at the Human Evaluation Round is published. Teams tune their solutions on the 1st Dataset.|
|12th of November, 2017||Submission of conversational agents is closed. Teams submit final solutions for the NIPS Live Competition.|
|20th of November - 3rd of December, 2017||Pre-NIPS Human Evaluation. Teams and volunteers start to evaluate solutions of teams. Evaluation is done remotely via a messaging platform.|
|4th-9th of December, 2017||Conversational Intelligence Live Competition at NIPS. Teams, conference participants and volunteers continue to evaluate solutions of teams. Announcement of winners.|
The competition consists of four rounds.
Both human evaluators and dialogue agents complete the same task.
Dataset collected during competition will be distributed under MIT license.
Participants have to make their systems available open source following the end of the competition, as the goal of the challenge is to enable the community to make sustained progress towards the creation of conversational intelligence. The data used by teams to pretrain models should be made open access or available on request for academic purposes.
To be announced.
Mikhail Burtsev, Valentin Malykh, MIPT, Moscow
Ryan Lowe, McGill University, Montreal
Iulian Serban, Yoshua Bengio, University of Montreal, Montreal
Alexander Rudnicky, Alan W. Black, Shrimai Prabhumoye, Carnegie Mellon University, Pittsburgh
Please contact firstname.lastname@example.org for additional information.
Please send your requests to email@example.com