Devendra Singh Chaplot
I am a member of the Founding Team at Thinking Machines Lab where I work on Research + Product. Previously, I was part of the Founding Team at Mistral AI, where I worked on training Mistral 7B, Mixtral 8x7B & Mistral Large, and led the Multimodal research team that trained Pixtral 12B and Pixtral Large. Earlier I was a Reasearch Scientist at Facebook AI Research where I worked at the intersection of Computer Vision and Robotics. I received my Ph.D. at the Machine Learning Department, School of Computer Science, Carnegie Mellon University. During my Ph.D., I worked with Prof. Ruslan Salakhutdinov and Prof. Abhinav Gupta on Building Intelligent Autonomous Navigation Agents. Prior to joining CMU, I graduated from IIT Bombay, India with a B.Tech in Computer Science & Engineering and a minor in Applied Statistics in August 2014. I also worked for about a year at Samsung Electronics HQ in South Korea.
CV | Google Scholar | Github | BioEmail: dc[at]thinkingmachines.ai
Ph.D. Thesis
Building Intelligent Autonomous Navigation AgentsPDF | Talk | Slides
Updates
Feb '25
Incredibly fortunate and excited to be part of
the founding team at Thinking Machines Lab!
Jan '23
Paper on Multi-skill Mobile Manipulation (M3)
accepted to ICLR 2023 as spotlight!
Dec '22
Our Multi-skill Mobile Manipulation (M3) model
wins the Habitat Rearrangement Challenge at NeurIPS 2022!
Oct '22
Released the HM3D-Semantics Dataset, the
largest public dataset of real-world 3D spaces with dense semantic annotations!
Jun '22
Co-organizing the Embodied AI Workshop and
Habitat ObjectNav Challenge at CVPR 2022
[Webpage]
Aug '21
For students applying for graduate
fellowships, I uploaded my research statement from the 2020 Facebook Fellowship
application.
[PDF]
Apr '21
Excited to join Facebook AI Research as a
Research Scientist!
Mar '21
I successfully defended my Ph.D. Thesis on
"Building Intelligent Autonomous Navigation Agents".
Jul '20
Jun '20
We won the CVPR 2020 Habitat ObjectNav
Challenge! Thanks Google for $10,000 GCP Credits.
Summer '20
Excited to intern with Jitendra Malik and
Deepak Pathak at Facebook AI Research!
Apr '20
Dec '19
Received the Nvidia Graduate Fellowship!
(Declined)
Dec '19
New paper on Learning to Explore using Active
Neural SLAM accepted at ICLR-20.
Oct '19
We received the Facebook Research Award for
PyRobot: Democratizing Robotics!
Jun '19
We won the CVPR-19 Habitat Navigation
Challenge! Thanks Google for $20,000 GCP Credits.
Oct '18
Excited to be a Workflow chair for ICML 2019.
Jun '18
Gave a talk at AIED-18 for our paper on
Learning Cognitive Models using Neural Networks.
[PDF]
Jun '18
Paper on end-to-end global pose estimation
using Neural Graph Optimization received Best Paper Award at the CVPR-18 Deep Learning
for Visual SLAM workshop!
[PDF]
Feb '18
New paper on our work at Apple AI Research on
end-to-end global pose estimation using Neural Graph Optimization posted on arxiv.
[PDF]
Jan '18
Released code, environment and pre-trained
model for AAAI-18 Language Grounding paper.
[Code]
Dec '17
Released code and pre-trained models for our
Doom AI agent, Arnold. Play against our Doom AI agent which won the Visual Doom AI
Competition 2017 Full Deathmatch.
[Code]
Nov '17
Two papers accepted for oral presentation at
AAAI-18 on WSD and Language Grounding.
Sep '17
Our Doom AI agent, Arnold won the Visual Doom
AI Competition 2017 Full Deathmatch.
Jun '17
MIT TechReview and Inverse write about our
recent paper on language grounding!
Select Research Projects
Navigating to Objects in the Real World
Theo Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, Devendra Singh Chaplot. (2022)
arXiv preprint arXiv:2212.00922
PDF | Webpage | Talk | Slides
SEAL: Self-supervised Embodied Active Learning
Devendra Singh Chaplot, Murtaza Dalal, Saurabh Gupta, Jitendra Malik, Ruslan Salakhutdinov. (2021)
Neural Information Processing Systems (NeurIPS-21)
PDF | Webpage | Talk | Slides
Goal-Oriented Semantic Exploration
Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, Ruslan Salakhutdinov. (2020)
Neural Information Processing Systems (NeurIPS-20)
PDF | Webpage | Code | Pre-trained models | Talk | Slides
Media: TechCrunch, Gizmodo, The Hindu, CMU News
Winner CVPR 2020 Habitat ObjectNav Challenge.
Neural Topological SLAM
Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, Saurabh Gupta. (2020)
Computer Vision and Pattern Recognition (CVPR-20), Seattle, USA
PDF | Webpage | Talk | Slides
Active Neural SLAM
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta,
Ruslan Salakhutdinov. (2020)
8th International Conference on Learning Representations (ICLR-20), Addis Adaba, Ethiopia
PDF | Webpage | Code | Pre-trained models | Talk | Slides | Blog
Media: VentureBeat, Synced, Analytics India Mag
Winner CVPR 2019 AI Habitat Navigation Challenge: 1st place (RGBD), joint 1st place (RGB)
Learning to follow language instructions
Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, Ruslan Salakhutdinov. (2018)
32nd AAAI Conference on Artificial Intelligence (AAAI-18), New Orleans, USA.
PDF | Code | Environment | Pre-trained models | Webpage
Media: MIT TechReview, Inverse
Learning to play deathmatches in Doom
Guillaume Lample*, Devendra Singh Chaplot*. (2017)
31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, USA.
Devendra Singh Chaplot*, Guillaume Lample*. (2017)
31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, USA. (demo)
PDF | Code | Pre-trained models | Demo videos (300K+ views)
Media: TechCrunch, Popular Science, Engadget, Daily Mail, Salon, Kotaku, ScienceAlert, Pittsburgh Post-Gazette, Inverse, CMU News
First place Visual Doom AI Competition 2017 Full Deathmatch
Second place Visual Doom AI Competition 2016 Full Deathmatch
See Google Scholar for a complete list of publications.
Talks
Building Intelligent Autonomous
Navigation Agents
Ph.D. Thesis DefenseVideo, Slides
Semantic Curiosity for Active Visual
Learning
ECCV 2020 (spotlight)Video, Slides
Object Goal Navigation using
Goal-Oriented Semantic Exploration
CVPR 2020 Embodied AI WorkshopWinning entry for Habitat ObjectNav Challenge
Video, Slides
Neural Topological SLAM for Visual
Navigation
CVPR 2020 Main ConferenceCVPR 2020 Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics
Video, Slides
Learning to Explore using Active
Neural SLAM
ICLR 2020 CVPR 2019 Habitat Embodied Agents Workshop, Winning entry
Video, Slides
Tutorial on Deep Reinforcement
Learning
2019 Summer Workshop on Machine Learning, Tepper School of Business, CMU, PittsburghWorkshop, Google Colab Notebook
Playing FPS Games with Deep
Reinforcement Learning
Nvidia GTC 2018Video, Slides
Doom and Unreal Game
Engines
Embodied Agents and Environments Workshop 2018, Facebook AI Research, Menlo ParkSlides
Gated-Attention Architectures for
Task-Oriented Language Grounding
AAAI 2018, oralSlides
Knowledge-based Word Sense
Disambiguation using Topic Models
AAAI 2018, oralSlides
Active Neural
Localization
NIPS 2017, Deep Reinforcement Learning SymposiumVideo, Slides