Devendra Singh Chaplot

I am a Research Scientist at Mistral AI working on building the next generation of AI models. Earlier I was a Reasearch Scientist at Facebook AI Research where I worked at the intersection of Computer Vision and Robotics. I received my Ph.D. at the Machine Learning Department, School of Computer Science, Carnegie Mellon University. During my Ph.D., I worked with Prof. Ruslan Salakhutdinov on Building Intelligent Autonomous Navigation Agents. I also worked closely with Abhinav Gupta and Saurabh Gupta. Prior to joining CMU, I graduated from IIT Bombay, India with a B.Tech in Computer Science & Engineering and a minor in Applied Statistics in August 2014, where I worked with Prof. Pushpak Bhattacharyya. I also worked for about a year at Samsung Electronics HQ in South Korea.

CV | Google Scholar | Github | Bio

Follow @dchaplot

Email: dc[at]mistral.ai

Ph.D. Thesis

Building Intelligent Autonomous Navigation Agents
PDF | Talk | Slides

Updates

Dec '23

Announced Mistral La Plateforme, API for three chat models and one emedding model.

[Webpage]

Dec '23

Released Mistral 8x7B, high-quality sparse mixture of experts model, Apache 2.0 .

[Webpage]

Oct '23

Mistral 7B paper is on Arxiv.

[PDF]

Sep '23

Released Mistral 7B, the best 7B model to date, Apache 2.0 .

[Webpage]

Mar '23

New ICCV Paper on Navigating to Objects specified by Images.

[PDF] [Webpage]

Feb '23

New Science Robotics paper on Navigating to Objects in the Real World.

[PDF] [Webpage]

Jan '23

Paper on Multi-skill Mobile Manipulation (M3) accepted to ICLR 2023 as spotlight!

[PDF] [Webpage]

Dec '22

Our Multi-skill Mobile Manipulation (M3) model wins the Habitat Rearrangement Challenge at NeurIPS 2022!

[Challenge] [Code]

Oct '22

Released the HM3D-Semantics Dataset, the largest public dataset of real-world 3D spaces with dense semantic annotations!

[PDF] [Webpage]

Oct '22

New paper on Retrospectives on the Embodied AI Workshop .

[PDF]

Oct '22

New paper on Instance-Specific Image Goal Navigation task.

[PDF]

Jun '22

Co-organizing the Embodied AI Workshop and Habitat ObjectNav Challenge at CVPR 2022

[Webpage]

Feb '22

PONI accepted to CVPR 2022 as oral!

[PDF] [Webpage]

Jan '22

FILM accepted to ICLR 2022

[PDF] [Webpage]

Sep '21

3 papers accepted at NeurIPS 2021: SEAL, NRNS, Habitat 2.0!

[Publications]

Aug '21

For students applying for graduate fellowships, I uploaded my research statement from the 2020 Facebook Fellowship application.

[PDF]

Jul '21

New ICML 2021 paper on Differentiable Spatial Planning using Transformers.

[PDF] [Webpage]

Apr '21

Excited to join Facebook AI Research as a Research Scientist!

Apr '21

Released code and pre-trained models for NeurIPS-20 paper on Object Goal Navigation.

[Code]

Mar '21

I successfully defended my Ph.D. Thesis on "Building Intelligent Autonomous Navigation Agents".

[Recording] [Slides]

Sep '20

Paper on Object Goal Navigation accepted at NeurIPS-20.

[PDF] [Webpage]

Jul '20

Our work on Object Goal Navigation was featured in TechCrunch, Gizmodo, The Hindu & CMU News.

[PDF] [Webpage]

Jun '20

We won the CVPR 2020 Habitat ObjectNav Challenge! Thanks Google for $10,000 GCP Credits.

[PDF] [Talk] [Slides]

Summer '20

Excited to intern with Jitendra Malik and Deepak Pathak at Facebook AI Research!

Apr '20

Our work on Active Neural SLAM was featured in VentureBeat, Synced and Analytics India Mag.

Apr '20

Released code and pre-trained models for ICLR-20 Active Neural SLAM paper.

[Code]

Feb '20

Paper on Neural Topological SLAM for Visual Navigation accepted at CVPR-20.

[PDF] [Webpage]

Jan '20

Received the Facebook Graduate Fellowship!

[Link] [CMU News]

Dec '19

Received the Nvidia Graduate Fellowship! (Declined)

Dec '19

New paper on Learning to Explore using Active Neural SLAM accepted at ICLR-20.

[PDF] [Webpage]

Oct '19

We received the Facebook Research Award for PyRobot: Democratizing Robotics!

Jun '19

We won the CVPR-19 Habitat Navigation Challenge! Thanks Google for $20,000 GCP Credits.

[Challenge] [PDF]

Summer '19

Interned with Abhinav Gupta and Saurabh Gupta at Facebook AI Research, Pittsburgh.

[PDF]

Feb '19

New paper on Embodied Multimodal Multitask Learning posted on arxiv.

[PDF] [Videos]

Oct '18

Excited to be a Workflow chair for ICML 2019.

Jun '18

Gave a talk at AIED-18 for our paper on Learning Cognitive Models using Neural Networks.

[PDF]

Summer '18

Interned with Dhruv Batra and Devi Parikh at Facebook AI Research, Menlo Park.

[PDF]

Jun '18

New AIED-18 paper on Learning Cognitive Models using Neural Networks posted on arxiv.

[PDF]

Jun '18

Paper on end-to-end global pose estimation using Neural Graph Optimization received Best Paper Award at the CVPR-18 Deep Learning for Visual SLAM workshop!

[PDF]

Jun '18

New ICML-18 paper on Gated Path Planning Networks posted on arxiv.

[PDF] [Code]

Apr '18

Released code, environment and pre-trained models for ICLR-18 Localization paper.

[Code]

Mar '18

Gave a talk on DeepRL in 3D environments at the Nvidia GTC 2018.

[Video] [Slides]

Feb '18

New paper on our work at Apple AI Research on end-to-end global pose estimation using Neural Graph Optimization posted on arxiv.

[PDF]

Feb '18

Gave two talks at AAAI-18, on WSD and Language Grounding.

[Slides-WSD] [Slides-LG]

Jan '18

Paper on Active Neural Localization accepted at ICLR-18.

[PDF] [Slides]

Jan '18

Released code, environment and pre-trained model for AAAI-18 Language Grounding paper.

[Code]

Jan '18

AAAI-18 WSD paper available on arXiv.

[PDF]

Dec '17

Released code and pre-trained models for our Doom AI agent, Arnold. Play against our Doom AI agent which won the Visual Doom AI Competition 2017 Full Deathmatch.

[Code]

Dec '17

Gave a talk on Active Neural Localization at the NIPS-17 DeepRL Symposium.

[Video] [Slides]

Nov '17

Two papers accepted for oral presentation at AAAI-18 on WSD and Language Grounding.

[PDF-WSD] [PDF-LG]

Sep '17

Our Doom AI agent, Arnold won the Visual Doom AI Competition 2017 Full Deathmatch.

[Competition Results]

Jun '17

MIT TechReview and Inverse write about our recent paper on language grounding!

[MIT TechReview] [Inverse]

Jun '17

New paper on Language Grounding posted on arXiv.

[PDF]

Select Research Projects

Navigating to Objects in the Real World

Navigating to Objects in the Real World
Theo Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, Devendra Singh Chaplot. (2022)
arXiv preprint arXiv:2212.00922
PDF | Webpage | Talk | Slides

SEAL: Self-supervised Embodied Active Learning

SEAL: Self-supervised Embodied Active Learning
Devendra Singh Chaplot, Murtaza Dalal, Saurabh Gupta, Jitendra Malik, Ruslan Salakhutdinov. (2021)
Neural Information Processing Systems (NeurIPS-21)
PDF | Webpage | Talk | Slides

Goal-Oriented Semantic Exploration

Object Goal Navigation using Goal-oriented Semantic Exploration
Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, Ruslan Salakhutdinov. (2020)
Neural Information Processing Systems (NeurIPS-20)
PDF | Webpage | Code | Pre-trained models | Talk | Slides
Media: TechCrunch, Gizmodo, The Hindu, CMU News
Winner CVPR 2020 Habitat ObjectNav Challenge.

Neural Topological SLAM

Neural Topological SLAM for Visual Navigation
Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, Saurabh Gupta. (2020)
Computer Vision and Pattern Recognition (CVPR-20), Seattle, USA
PDF | Webpage | Talk | Slides

Active Neural SLAM

Learning to Explore using Active Neural SLAM
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta,
Ruslan Salakhutdinov. (2020)
8th International Conference on Learning Representations (ICLR-20), Addis Adaba, Ethiopia
PDF | Webpage | Code | Pre-trained models | Talk | Slides | Blog
Media: VentureBeat, Synced, Analytics India Mag
Winner CVPR 2019 AI Habitat Navigation Challenge: 1st place (RGBD), joint 1st place (RGB)

Learning to follow language instructions

Gated-Attention Architectures for Task-Oriented Language Grounding
Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, Ruslan Salakhutdinov. (2018)
32nd AAAI Conference on Artificial Intelligence (AAAI-18), New Orleans, USA.
PDF | Code | Environment | Pre-trained models | Webpage
Media: MIT TechReview, Inverse

Learning to play deathmatches in Doom

Playing FPS Games with Deep Reinforcement Learning
Guillaume Lample*, Devendra Singh Chaplot*. (2017)
31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, USA.

Arnold: An Autonomous Agent to play FPS Games (Best Demo Award)
Devendra Singh Chaplot*, Guillaume Lample*. (2017)
31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, USA. (demo)

PDF | Code | Pre-trained models | Demo videos (300K+ views)
Media: TechCrunch, Popular Science, Engadget, Daily Mail, Salon, Kotaku, ScienceAlert, Pittsburgh Post-Gazette, Inverse, CMU News
First place Visual Doom AI Competition 2017 Full Deathmatch
Second place Visual Doom AI Competition 2016 Full Deathmatch

See Google Scholar for a complete list of publications.

Devendra Singh Chaplot

Ph.D. Thesis

Updates

Select Research Projects

Navigating to Objects in the Real World

SEAL: Self-supervised Embodied Active Learning

Goal-Oriented Semantic Exploration

Neural Topological SLAM

Active Neural SLAM

Learning to follow language instructions

Learning to play deathmatches in Doom

Talks

Building Intelligent Autonomous Navigation Agents

Semantic Curiosity for Active Visual Learning

Object Goal Navigation using Goal-Oriented Semantic Exploration

Neural Topological SLAM for Visual Navigation

Learning to Explore using Active Neural SLAM

Tutorial on Deep Reinforcement Learning

Playing FPS Games with Deep Reinforcement Learning

Doom and Unreal Game Engines

Gated-Attention Architectures for Task-Oriented Language Grounding

Knowledge-based Word Sense Disambiguation using Topic Models

Active Neural Localization