Shivansh Patel

I am a Ph.D. student in Computer Science at UIUC advised by Svetlana Lazebnik. I also closely collaborate with Yunzhu Li. Earlier, I graduated with a B.Tech. in Electrical Engineering from IIT Kanpur. I am broadly interested in Robotics, Reinforcement Learning, and Computer Vision.

Previously, I was a visiting scholar in GrUVi lab at Simon Fraser University working with Manolis Savva and Angel X. Chang. During my undergraduate stay at IIT Kanpur, I worked with Vinay P. Namboodiri on attention, explainability and uncertaintly in Vision and Language.

CV | E-Mail | Google Scholar | Github | Twitter

Mentoring

I am always interested in mentoring motivated undergraduates and masters in CS/ECE/ME. If you want to discuss project ideas/research opportunities, please feel free to send me an email. I am currently working on robot learning, with a focus on leveraging foundation models to achieve open-world manipulation capabilities.

Publications

A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
Shivansh Patel*, Xinchen Yin, Wenlong Huang, Shubham Garg, Hooshang Nayyeri, Li Fei-Fei, Svetlana Lazebnik, Yunzhu Li
CoRL LEAP and LangRob Workshop 2024
paper | project

Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environments
Sonia Raychaudhuri, Saim Wani, Shivansh Patel, Unnat Jain, Angel X. Chang
EMNLP 2021
paper | project | code

Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
Shivansh Patel*, Saim Wani*, Unnat Jain*, Alexander Schwing, Svetlana Lazebnik, Manolis Savva, Angel X. Chang
ICCV 2021
paper | project | code

MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation
Saim Wani*, Shivansh Patel*, Unnat Jain*, Angel X. Chang, Manolis Savva
NeurIPS 2020
paper | project | code

Robust Explanations for Visual Question Answering
Badri N. Patro, Shivansh Patel, Vinay P. Namboodiri
WACV 2020
paper | code

U-cam: Visual explanation using uncertainty based class activation maps
Badri N. Patro, Mayank Lunayach, Shivansh Patel, Vinay P. Namboodiri
ICCV 2019
paper | project | code

Granular Multimodal Attention Networks for Visual Dialog
Badri N. Patro, Shivansh Patel, Vinay P. Namboodiri
ICCV ISV Workshop 2019
paper

Affiliations

IIT Kanpur
2016-2020

FreshGravity
Summer 2019

SFU
Jan-July 2020

UIUC
Fall 2021 - Present

News

Oct 2024	Paper on keypoint rewards accepted in CoRL 2024 LEAP and LangRob workshop.
Sept 2021	Serving as reviewer for ECCV 2022, CVPR 2022, and TPAMI.
Sept 2021	Paper on Language-Aligned Waypoints in VLN accepted in EMNLP 2021.
Sept 2021	Serving as program committee in EXplainable AI approaches for debugging and diagnosis workshop in NeurIPS 2021.
July 2021	Paper on emergent communication in embodied agents accepted at ICCV 2021.
Jun 2021	Panelist in a discussion on navigation agents at Embodied AI Workshop, CVPR 2021.
Apr 2021	Accepted UIUC CS Ph.D. offer! Will join Fall 2021.
Jan 2021	Hosting the MultiON challenge at Embodied AI Workshop, CVPR 2021.
Sept 2020	Paper on Multi-Object Navigation got accepted at NeurIPS 2020.
May 2020	Named finalist for Qualcomm Innovation Fellowship India 2020.
Jan 2020	Joining Simon Fraser University as a visiting researcher.
Feb 2020	Paper on Explanations for Visual Question Answering got accepted at WACV 2020.
Oct 2019	Paper on Visual Explanation got accepted at ICCV 2019.
Oct 2019	Paper on Multimodal Attention Network got accepted at ICCV ISV Workshop 2019.
Sept 2018	Received Academic Excellence Award for meritorious performance at IIT Kanpur.

Template credits: Deepak and Unnat