About

I am a Research Scientist at Meta Superintelligence Labs (MSL), ex-FAIR Perception. I completed my Ph.D. at Georgia Tech, advised by Prof. Patricio A. Vela in IVALab. Prior to that, I received M.S. from EECS at University of Michigan (UM) in 2014, and B.S. from EE at National Taiwan University (NTU) in 2011.
Building on my PhD work in robotics, I enjoy making intelligent machines interact with the physical world through frontier VLMs, 3D vision, and world action models.

Publications

Please see my Google Scholar for complete publication list.
SAM 3D: 3Dfy Anything in Images
SAM 3D Team, Xingyu Chen*, Fu-Jen Chu*, Pierre Gleize*, Kevin J Liang*, Alexander Sax*, Hao Tang*, Weiyao Wang*, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, Jiawei Liu, Ziqi Ma, Anushka Sagar, Bowen Song, Xiaodong Wang, Jianing Yang, Bowen Zhang, Piotr Dollar, Georgia Gkioxari, Matt Feiszli, Jitendra Malik (*core contributors)
CVPR 2026 (Best Paper Honorable Mention)
Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models
Runsen Xu, Weiyao Wang, Hao Tang, Xingyu Chen, Xiaodong Wang, Fu-Jen Chu, Matt Feiszli, Kevin J Liang
CVPR 2026
OmniPose6D: Towards Short-Term Object Pose Tracking in Dynamic Scenes from Monocular RGB
Yunzhi Lin, Yipu Zhao, Fu-Jen Chu, Xingyu Chen, Weiyao Wang, Hao Tang, Patricio A Vela, Matt Feiszli, Kevin J Liang
IROS 2025
HOIGPT: Learning Long-Sequence Hand-Object Interaction with Language Models
Mingzhen Huang, Fu-Jen Chu, Bugra Tekin, Kevin J Liang, Haoyu Ma, Weiyao Wang, Xingyu Chen, Pierre Gleize, Hongfei Xue, Siwei Lyu, Kris Kitani, Matt Feiszli, Hao Tang
CVPR 2025
Propose, Assess, Search: Harnessing LLMs for Goal-Oriented Planning in Instructional Videos
Md Mohaiminul Islam, Tushar Nagarajan, Huiyu Wang, Fu-Jen Chu, Kris Kitani, Gedas Bertasius, Xitong Yang
ECCV 2024 (Oral)
EgoSG: Learning 3D Scene Graphs from Egocentric RGB-D Sequences
Chaoyi Zhang, Xitong Yang, Ji Hou, Kris Kitani, Weidong Cai, Fu-Jen Chu
CVPR workshop 2024
Ego-Exo4D: Understanding Skilled Human Activity from First-and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Fu-Jen Chu* et al.
CVPR 2024 (Oral)
HyperMix: Out-of-Distribution Detection and Classification in Few-Shot Settings
Nikhli Mehta*, Kevin J Liang*, Jing Huang, Fu-Jen Chu, Li Yin, Tal Hassner
WACV 2024
Relational Space-Time Query in Long-Form Videos
Xitong Yang, Fu-Jen Chu, Matt Feiszli, Raghav Goyal, Lorenzo Torresani, Du Tran
CVPR 2023 highlights
Primitive Shape Recognition for Object Grasping
Yunzhi Lin, Chao Tang, Fu-Jen Chu, Patricio A. Vela
submitted to IJRR
Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks
Fu-Jen Chu, Ruinian Xu, Chao Tang, Patricio A. Vela
submitted to IJRR
An Affordance Keypoint Detection Network for Robot Manipulation
Ruinian Xu, Fu-Jen Chu, Chao Tang, Weiyu Liu, Patricio A. Vela
IEEE RA-L 2021 with ICRA 2021
GKNet: Grasp Keypoint Network for Grasp Detection
Ruinian Xu, Fu-Jen Chu, Patricio A. Vela
IJRR 2021
Improving Vision-Based Robotic Manipulation with Affordance Understanding
Fu-Jen Chu
Ph.D. Dissertation at Georgia Institute of Technology 2020
Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping
Yunzhi Lin*, Chao Tang*, Fu-Jen Chu, Patricio A. Vela
IEEE ICRA 2020
Toward Affordance Detection and Ranking on Novel Objects for Real-world Robotic Manipulation
Fu-Jen Chu, Ruinian Xu, Landan Seguin, Patricio A. Vela
IEEE RA-L 2019 with IROS 2019
Learning Affordance Segmentation for Real-world Robotic Manipulation via Synthetic Images
Fu-Jen Chu, Ruinian Xu, Patricio A. Vela
IEEE RA-L 2019 with ICRA 2019
Real-World, Multiobject, Multigrasp Detection
Fu-Jen Chu, Ruinian Xu, Patricio A. Vela
IEEE RA-L 2018 with IROS 2018
Hands-Free Control of an Assistive Manipulator Using Augmented Reality and Tongue Drive System
Fu-Jen Chu, Ruinian Xu, Zhenxuan Zhang, Patricio A. Vela, Maysam Ghovanloo
IEEE IROS 2018
The Helping Hand: An Assitive Manipulation Framework Using Augmented Reality and Tongue-Drive Interfaces
Fu-Jen Chu, Ruinian Xu, Zhenxuan Zhang, Patricio A. Vela, Maysam Ghovanloo
IEEE EMBC 2018
When Crowdsourcing Meets Mobile Sensing: A Social Network Perspective
Pin-Yu Chen, Shin-Ming Cheng, Chia-Wei Lien, Pai-Shun Ting, Fu-Jen Chu
IEEE Communication Magazine 2015
When Crowdsourcing Meets Mobile Sensing: A Social Network Perspective
Pin-Yu Chen, Chia-Wei Lien, Fu-Jen Chu, Pai-Shun Ting, Shin-Ming Cheng
IEEE Globecom Workshops 2015

Industry

Facebook AI Research (FAIR)

Research Scientist, Aug. 2020 - present

Texas Instruments

Research Intern, Summer 2016

with Dr. Murtaza Ali

Volvo Group

Research Intern, Spring 2016

with Dr. Fares Beainy

Education

Georgia Institute of Technology

Doctor of Philosophy

Electrical and Computer Engineering

University of Michigan

Master of Science

Electrical Engineering and Computer Science

National Taiwan University

Bachelor of Science

Electrical Engineering