|
Mike Roberts |
Biography
I'm a senior research scientist at Adobe Research, where I lead the development of the SPEAR simulator. I'm interested in using photorealistic synthetic data for embodied AI and computer vision. Before joining Adobe Research, I was a research scientist in the Intelligent Systems Lab at Intel. Before joining Intel, I was a research scientist at Apple, where I led the development of the Hypersim dataset.
In 2019, I received my PhD from Stanford University, where I was advised by Pat Hanrahan. My dissertation work was at the intersection of computer graphics, robotics, and computer vision, where I focused on enabling drones to scan 3D environments and execute cinematic camera trajectories. During my graduate studies, I interned at Microsoft Research and Skydio. Before attending Stanford, I was a research fellow at Harvard University, where I was advised by Hanspeter Pfister. In 2012, I worked with John Owens and David Luebke to develop the Introduction to Parallel Programming course at Udacity. In 2009, I interned at NVIDIA.
For an overview of my dissertation work, see this talk from TEDxBerkeley 2017.
Prospective interns: If you're interested in interning with me, please email me with a link to your website and a short description of what you'd like to work on.
Selected Publications
A complete listing of my publications is available on Google Scholar.
|
SPEAR: A Simulator for Photorealistic Embodied AI Research |
|
PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Drop |
|
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding |
|
Trajectory Optimization Methods for Drone Cameras |
|
Submodular Trajectory Optimization for Aerial 3D Scanning |
|
Generating Dynamically Feasible Trajectories for Quadrotor Cameras
Featured in the Highlights of SIGGRAPH session at the FMX Festival 2017 |
|
An Interactive Tool for Designing Quadrotor Camera Shots Featured in the SIGGRAPH Asia 2015 Technical Papers Trailer
|
|
Saturated Reconstruction of a Volume of Neocortex
|
|
Large-Scale Automatic Reconstruction of Neuronal Processes from Electron Microscopy Images |
|
Design and Evaluation of Interactive Proofreading Tools for Connectomics |
|
Neural Process Reconstruction from Sparse User Scribbles |
|
A Work-Efficient GPU Algorithm for Level Set Segmentation |
Software and Datasets
SPEAR: A Simulator for Photorealistic Embodied AI Research
|
Interactive simulators have become powerful tools for training embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited generality, programmability, and rendering speed. We address these limitations by introducing SPEAR: A Simulator for Photorealistic Embodied AI Research. At its core, SPEAR is a Python library that can connect to, and programmatically control, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions to Python, representing an order-of-magnitude increase in programmable functionality over existing UE-based simulators. Additionally, a single SPEAR instance can render 1920x1080 photorealistic beauty images directly into a user's NumPy array at 73 frames per second - an order of magnitude faster than existing UE plugins - while also providing ground truth image modalities that are not available in any existing UE-based simulator (e.g., a non-diffuse intrinsic image decomposition, material IDs, and physically based shading parameters). Finally, SPEAR introduces an expressive high-level programming model that enables users to specify complex graphs of UE work with arbitrary data dependencies among work items, and to execute these graphs deterministically within a single UE frame.
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
|
For many fundamental scene understanding tasks, it is difficult or impossible to obtain per-pixel ground truth labels from real images. We address this challenge by introducing Hypersim, a photorealistic synthetic dataset for holistic indoor scene understanding. To create our dataset, we leverage a large repository of synthetic scenes created by professional artists, and we generate 77,400 images of 461 indoor scenes with detailed per-pixel labels and corresponding ground truth geometry. Our dataset: (1) relies exclusively on publicly available 3D assets; (2) includes complete scene geometry, material information, and lighting information for every scene; (3) includes dense per-pixel semantic instance segmentations and complete camera information for every image; and (4) factors every image into diffuse reflectance, diffuse illumination, and a non-diffuse residual term that captures view-dependent lighting effects.
Students
I am fortunate to work with amazing students: Habib Slim (KAUST)
Personal
Before going to Stanford, I used to DJ in front of hundreds of people every weekend. I was a resident at The Republik and The Bamboo Tiki Room in Calgary, Canada. The Republik and the Bamboo were voted the 2nd and 3rd best places to dance in the FFWD Best of Calgary 2010. More recently, I won a national DJing competition to perform at Glowchella 2013 in San Francisco, and I have subsequently opened for The Chainsmokers, Martin Solveig, and several other notable international artists.
I think of my sound as a funky chunky soul stomp bigbeat boogaloo mashup of timeless dance music, i.e., imagine what it would sound like if James Brown, Ray Charles, The Beatles, Fatboy Slim, and Daft Punk all took acid together and played a sweaty warehouse primetime party set at the Apollo Theater Harlem NYC circa 1969. You can listen to a mix here. Note that Boogaloo is a genre of Latin dance music from the 1960s.