Fanyi Xiao

Email: fyxiao at ucdavis dot edu

I am an Applied Scientist working on computer vision and machine learning at Amazon AI. Previously, I finished my PhD at the University of California Davis, advised by Prof. Yong Jae Lee. Before that, I completed my master's degree in Robotics at Carnegie Mellon University.

During my PhD, I am very fortunate to have spent time at Disney Research working with Prof. Leonid Sigal, at NVIDIA Research with Dr. Xiaodong Yang and Dr. Ming-Yu Liu, and at Facebook AI Research (FAIR) with Dr. Christoph Feichtenhofer, Prof. Kristen Grauman and Prof. Jitendra Malik.

I'm mostly interested in multimodal learning with minimal human supervision as well as video understanding. Here is a talk I gave recently.

CV / Scholar / Github

News

09/20 -- Our paper on adaptive anti-aliasing won best paper award at BMVC 2020!
08/20 -- I have recently graduated and joined the Amazon AI team!
11/19 -- YOLACT won Most Innovative Award at COCO Object Detection Challenge, ICCV 2019.
07/19 -- Two papers (one oral one poster) get accepted at ICCV 19.
05/19 -- I will be joining Facebook AI Research for a summer internship!
12/18 -- Code for our video object detection work is now available on Github. Give it a try!
07/18 -- Our work on video object detection is accepted at ECCV 18, see you in Munich!
06/18 -- I am awarded a Best Graduate Researcher Award by CS Dept. of UC Davis.

Papers

MoDist: Motion Distillation for Self-supervised Video Representation Learning
Fanyi Xiao, Joseph Tighe, Davide Modolo
Surprising effectiveness of simple motion prior for video SSL

YolactEdge: Real-time Instance Segmentation on the Edge
Haotian Liu*, Rafael A. Rivera-Soto*, Fanyi Xiao, Yong Jae Lee
IEEE International Conference on Robotics and Automation (ICRA), 2020
[arXiv] [Code] [Talk] [Demo] [Colab Notebook]
Run instance segmentation on your Jetson device

Delving Deeper into Anti-aliasing in ConvNets
Xueyan Zou, Fanyi Xiao, Zhiding Yu, Yong Jae Lee
British Machine Vision Conference (BMVC), 2020
[Project] [Code] [Talk]
Best Paper Award

Audiovisual SlowFast Networks for Video Recognition
Fanyi Xiao, Yong Jae Lee, Kristen Grauman, Jitendra Malik, Christoph Feichtenhofer

YOLACT++: Better Real-time Instance Segmentation
Daniel Bolya*, Chong Zhou*, Fanyi Xiao, Yong Jae Lee
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)
YOLACT++ (v1.2) code released

Identity from here, Pose from there: Self-supervised Disentanglement and Generation of Objects using Unlabeled Videos
Fanyi Xiao, Haotian Liu, Yong Jae Lee
International Conference on Computer Vision (ICCV), 2019

YOLACT: Real-time Instance Segmentation
Daniel Bolya, Chong Zhou, Fanyi Xiao, Yong Jae Lee
International Conference on Computer Vision (ICCV), 2019
oral presentation
code available!

STEP: Spatio-Temporal Progressive Learning for Video Action Detection
Xitong Yang, Xiaodong Yang, Ming-Yu Liu, Fanyi Xiao, Larry Davis, Jan Kautz
Computer Vision and Pattern Recognition (CVPR), 2019
oral presentation

Video Object Detection with an Aligned Spatial-Temporal Memory
Fanyi Xiao and Yong Jae Lee
European Conference on Computer Vision (ECCV), 2018

Who Will Share My Image? Predicting the Content Diffusion Path in Online Social Networks
Wenjian Hu, Krishna Kumar Singh*, Fanyi Xiao*, Jinyoung Han, Chen-Nee Chuah and Yong Jae Lee
ACM International Conference on Web Search and Data Mining (WSDM), 2018
* equal contribution

Weakly-supervised Visual Grounding of Phrases with Linguistic Structures
Fanyi Xiao, Leonid Sigal and Yong Jae Lee
Computer Vision and Pattern Recognition (CVPR), 2017

Track and Segment: An Iterative Unsupervised Approach for Video Object Proposals
Fanyi Xiao and Yong Jae Lee
Computer Vision and Pattern Recognition (CVPR), 2016
spotlight presentation

Track and Transfer: Watching Videos to Simulate Strong Human Supervision for Weakly-Supervised Object Detection
Krishna Singh, Fanyi Xiao and Yong Jae Lee
Computer Vision and Pattern Recognition (CVPR), 2016

Discovering the Spatial Extent of Relative Attributes
Fanyi Xiao and Yong Jae Lee
International Conference on Computer Vision (ICCV), 2015
oral presentation

Efficient Model Evaluation with Bilinear Separation Model
Fanyi Xiao and Martial Hebert
Winter Conference on Applications of Computer Vision (WACV), 2015

Transitive Distance Clustering with K-Means Duality
Zhiding Yu, Chunjing Xu, Deyu Meng, Zhuo Hui, Fanyi Xiao, Wenbo Liu, Jianzhuang Liu
Computer Vision and Pattern Recognition (CVPR), 2014

Physical Querying with Multi-modal Sensing
Iljoo Baek, Taylor Stine, Denver Dash, Fanyi Xiao, Yaser Sheikh, Yair Movshovitz-Attias, Mei Chen, Martial Hebert, and Takeo Kanade
Winter Conference on Applications of Computer Vision (WACV), 2014

Industry Experience


Facebook AI Research (Summer 2019)
Developed an audiovisual network architecture for video understanding

NVIDIA Research (Summer 2017)
Developed a novel method for action detection

Disney Research (Summer 2016)
Developed a novel model for free-form language grounding on images