Zachary (Zak) Murez

I am currently a research scientist at Magic Leap.

I received my Ph.D. from the University of California, San Diego (UCSD) in Computer Science and Engineering studying computer vision, where I was advised by Profs David Kriegman and Ravi Ramamoorthi.

I received my Bachelor of Science degree in Intensive Mathematics and Computer Science from Yale University in 2011. I also took many upper division physics, chemistry, and biology courses.

Curriculum Vitae (CV)

Email: z...@cs.ucsd.edu

Atlas: End-to-End 3D Scene Reconstruction from Posed Images
Zak Murez, Tarrence van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, Andrew Rabinovich
ECCV 2020
Project | Paper | Code | Video

Abstract: We present an end-to-end 3D reconstruction method for a scene by directly regressing a truncated signed distance function (TSDF) from a set of posed RGB images. Traditional approaches to 3D reconstruction rely on an intermediate representation of depth maps prior to estimating a full 3D model of a scene. We hypothesize that a direct regression to 3D is more effective. A 2D CNN extracts features from each image independently which are then back-projected and accumulated into a voxel volume using the camera intrinsics and extrinsics. After accumulation, a 3D CNN refines the accumulated features and predicts the TSDF values. Additionally, semantic segmentation of the 3D model is obtained without significant computation. This approach is evaluated on the Scannet dataset where we significantly outperform state-of-the-art baselines (deep multiview stereo followed by traditional TSDF fusion) both quantitatively and qualitatively. We compare our 3D semantic segmentation to prior methods that use a depth sensor since no previous work attempts the problem with only RGB input.

Image to Image Translation for Domain Adaptation
Zak Murez, Soheil Kolouri, David Kriegman, Ravi Ramamoorthi, Kyungnam Kim
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Paper

Abstract: We propose a general framework for unsupervised domain adaptation, which allows deep neural networks trained on a source domain to be tested on a different target domain without requiring any training annotations in the target domain. This is achieved by adding extra networks and losses that help regularize the features extracted by the backbone encoder network. To this end we propose the novel use of the recently proposed unpaired image-toimage translation framework to constrain the features extracted by the encoder network. Specifically, we require that the features extracted are able to reconstruct the images in both domains. In addition we require that the distribution of features extracted from images in the two domains are indistinguishable. Many recent works can be seen as specific cases of our general framework. We apply our method for domain adaptation between MNIST, USPS, and SVHN datasets, and Amazon, Webcam and DSLR Office datasets in classification tasks, and also between GTA5 and Cityscapes datasets for a segmentation task. We demonstrate state of the art performance on each of these datasets.

Learning to See through Turbulent Water
Zhengqin Li, Zak Murez, David Kriegman, Ravi Ramamoorthi, Manmohan Chandraker
IEEE Winter Conf. on Applications of Computer Vision (WACV) 2018
Paper | Code | Data

Abstract: Imaging through dynamic refractive media, such as looking into turbulent water, or through hot air, is challenging since light rays are bent by unknown amounts leading to complex geometric distortions. Inverting these distortions and recovering high quality images is an inherently ill-posed problem, leading previous works to require extra information such as high frame-rate video or a template image, which limits their applicability in practice. This paper proposes training a deep convolution neural network to undistort dynamic refractive effects using only a single image. The neural network is able to solve this ill-posed problem by learning image priors as well as distortion priors. Our network consists of two parts, a warping net to remove geometric distortion and a color predictor net to further refine the restoration. Adversarial loss is used to achieve better visual quality and help the network hallucinate missing and blurred information. To train our network, we collect a large training set of images distorted by a turbulent water surface. Unlike prior works on water undistortion, our method is trained end-to-end only requires a single image and does not use a ground truth template at test time. Experiments show that by exploiting the structure of the problem, our network outperforms state-of-the-art deep image to image translation.

Depth and Image Restoration from Light Field In a Scattering Medium
Jiandong Tian, Zak Murez, Tong Cui, Zhen Zhang, David Kriegman, Ravi Ramamoorthi
IEEE International Conference on Computer Vision (ICCV) 2017
Paper

Abstract: Traditional imaging methods and computer vision algorithms are often ineffective when images are acquired in scattering media, such as underwater, fog, and biological tissue. Here, we explore the use of light field imaging and algorithms for image restoration and depth estimation that address the image degradation from the medium. Towards this end, we make the following three contributions. First, we present a new single image restoration algorithm which removes backscatter and attenuation from images better than existing methods do, and apply it to each view in the light field. Second, we combine a novel transmission based depth cue with existing correspondence and defocus cues to improve light field depth estimation. In densely scattering media, our transmission depth cue is critical for depth estimation since the images have low signal to noise ratios which significantly degrades the performance of the correspondence and defocus cues. Finally, we propose shearing and refocusing multiple views of the light field to recover a single image of higher quality than what is possible from a single view. We demonstrate the benefits of our method through extensive experimental results in a water tank.

Photometric Stereo in a Scattering Medium
Zak Murez, Tali Treibitz, Ravi Ramamoorthi, and David Kriegman
International Conference on Computer Vision (ICCV) 2015
Paper | Video

Abstract: Photometric stereo is widely used for 3D reconstruction. However, its use in scattering media such as water, biological tissue and fog has been limited until now, because of forward scattered light from both the source and object, as well as light scattered back from the medium (backscatter). Here we make three contributions to address the key modes of light propagation, under the common single scattering assumption for dilute media. First, we show through extensive simulations that single-scattered light from a source can be approximated by a point light source with a single direction. This alleviates the need to handle light source blur explicitly. Next, we model the blur due to the scattering of light from the object. We measure the object point-spread function and introduce a simple deconvolution method. Finally, we show how imaging fluorescence emission where available eliminates the backscatter component and increases the signal-to-noise ratio. Experimental results in a water tank, with different concentrations of scattering media added, show that deconvolution produces higher-quality 3D reconstructions than previous techniques, and that when combined with fluorescence, can produce results similar to that in clear water even for highly turbid media.

Maximizing the potential of electron cryomicroscopy data collected using direct detectors
David Veesler, Melody G Campbell, Anchi Cheng, Chi-yu Fu, Zachary Murez, John E Johnson, Clinton S Potter, Bridget Carragher
Journal of Structural Biology (JSB) 2013
Paper

Abstract: Single-particle electron cryo-microscopy is undergoing a technical revolution due to the recent developments of direct detectors. These new recording devices detect electrons directly (i.e. without conversion into light) and feature significantly improved detective quantum efficiencies and readout rates as compared to photographic films or CCDs. We evaluated here the potential of one such detector (Gatan K2 Summit) to enable the achievement of near-atomic resolution reconstructions of biological specimens when coupled to a widely used, mid-range transmission electron microscope (FEI TF20 Twin). Compensating for beam-induced motion and stage drift provided a 4.4 Å resolution map of Sulfolobus Turreted Icosahedral Virus (STIV), which we used as a test particle in this study. Several motion correction and dose fractionation procedures were explored and we describe their influence on the resolution of the final reconstruction. We also compared the quality of this data to that collected with a FEI Titan Krios microscope equipped with a Falcon I direct detector, which provides a benchmark for data collected using a high-end electron microscope.

Shape From Fluorescence
Tali Treibitz, Zak Murez, B. Greg Mitchell, and David Kriegman
European Conference on Computer Vision (ECCV) 2012
Paper | Video

Abstract: Beyond day glow highlighters and psychedelic black light posters, it has been estimated that fluorescence is a property exhibited by 20% of objects. When a fluorescent material is illuminated with a short wavelength light, it re-emits light at a longer wavelength isotropically in a similar manner as a Lambertian surface reflects light. This hitherto neglected property opens the doors to using fluorescence to reconstruct 3D shape with some of the same techniques as for Lambertian surfaces – even when the surface’s reflectance is highly non-Lambertian. Thus, performing reconstruction using fluorescence has advantages over purely Lambertian surfaces. Single image shape-from-shading and calibrated Lambertian photometric stereo can be applied to fluorescence images to reveal 3D shape. When performing uncalibrated photometric stereo, both fluorescence and reflectance can be used to recover Euclidean shape and resolve the generalized bas relief ambiguity. Finally for objects that fluoresce in wavelengths distinct from their reflectance (such as plants and vegetables), reconstructions do not suffer from problems due to inter-reflections. We validate these claims through experiments.

More personal stuff

Ultimate (previously known as Ultimate Frisbee) is a rapidly growing sport played across the world in pickup games and by recreational, school, club, professional and national teams. It was granted full IOC (International Olympic Committee) recognition on 2 Aug 2015, allowing for the possibility of it being competed at the Olympic Games in the future.

Ultimate is a non-contact field sport played with a plastic disk. The offensive team must progress the disk down the field and into the endzone by passing it amongst teammates. While holding the disk, the player may not take steps (must keep a pivot foot). Interceptions and incomplete passes are turnovers. See this video for highlights from the 2015 pro season.

Throughout high school and college I played some pickup ultimate, but after two years of graduate school, I found the UCSD Air Squids, fell in love, and started playing competitively. In 2014 we went to the USA Ultimate College National Championships and placed 13th in the nation. 2015 was a rebuilding year for the Squids, and we placed 5th at regionals. We hope to go back to nationals this year!

Parkour is the physical practice of traversing elements in both urban and rural settings. The goal is to move from one point to another as quickly and efficiently as possible, using only the human body and one's environment. Freerunning is a related discipline, but focuses on creativity and aesthetic beauty in one's movements. Freerunning includes parkour motions, but often also involves flips, twists, and spins.

While popular media portrays parkour as a group of daredevils jumping off rooftops, this is not representative of the sport. Similar to martial arts, parkour is a physical discipline, involving slow progression from novice to expert. One can think of parkour as training you for the flight while martial arts trains you for the fight. Extensive repetition of moves in a safe environment and conditioning minimize the risk. Most training is done close to the ground or in a gymnastics gym.

I practiced parkour on and off for three years, although my main athletic focus for most of that time was competitive swimming. I still enjoy going to gymnastic gyms and playing around, as well as doing parkour like conditioning. However, my main athletic focus currently is Ultimate Frisbee. Watch my 2010 showreel here!

Parkour is extremely fun and brings the kid that used to love to play on the playground back to life. When practiced safely, it is great physical and mental training, keeping the body, mind and spirit young. It is never too late to start.

Swimming was my main sport from age 9 through college. I started competitive swimming with the club Team Santa Monica. In 2003 I competed at the Pan-Am Maccabi Games in Santiago, Chile, and won four gold medals for Team USA. In 2005 I qualified for the USA Swimming Junior National Championships. From 2004-2007, I swam for Venice High School where I broke 5 school records. In 2009 I competed at the Maccabiah Games in Tel Aviv, Israel, where I finalled in the 100m breaststroke. From 2007-2011 I swam NCAA D1 for Yale University. After graduating from Yale, I retired from competitive swimming, although I still swim from time to time for cross training.

My sister, Andi Murez started competitive swimming soon after I did, and we trained together through high school. In 2009 she competed alongside me at the Maccabiah Games, winning multiple gold medals and breaking an Israeli national record. After graduating from Stanford, she decided to move to Israel to compete on their national team. She has since qualified for the 2016 Olympic Games where she will be competing this summer, and I will be in the stands cheering for her!

Class Projects

Nico (2010) The final project for Intelligent Robotics. Yale's humanoid robot, NICO, was scripted to play Tic Tac Toe against a subject. At one point, NICO cheats, and the subject's reaction was observed.

John Updike's A&P (2007) A reenactment of John Updike's A&P for a final project in AP English Literature.

Canterbury Tales - A Reeve's Tale (2006) My 11th grade English class was divided into groups and each group had to present a tale from the Canterbury Tales. We decided to do a 3D animated film.

Russian Revolution (2005) A reenactment of events during the Russian Revolution for a final project in AP European History.

VHS School Rules (2005) As a favor to the Principal, I made a video going over some of the important school rules to be shown at the orientation assembly.

Bio Bug (2004) The final project for my 9th grade Biology class was to, in small groups, invent our own organism and present it to the class. My partner and I decided to do it as a movie.

Dr. Zakell (2002) As a book report on "Dr. Jekyll and Mr. Hyde", I created my own reenactment.

Geothermal (2001) After my Science teacher heard about the video I had done for my History class, she wanted one, too. So, as an extra credit assignment, I made a short video explaining how geothermal energy is collected.

King Zak (2001) For my Ancient History class, I designed an Egyptian pyramid. I used the CAD program Vectorworks to create a 3D Model, which I animated in Cinema 4D. I then edited it into a video using Adobe Premiere.

Silly Projects

The 1000 (2010) A short film about the rivalry between two Yale distance swimmers.

The Night Before Yale Penn Dartmouth (2010) The swim team settling into the hotel, the night before the Yale-Penn-Dartmouth Tri-Meet.

Zaky Matrix (2003) A short film driven purely by action and special effects. It was a lot of fun and a great learning experience, but, unfortunately, was never finished.

A Day in the Life of Max (2003) My friends, Adam and Max, were over for the day. We decided to shoot this slapstick bathroom comedy. There was no planning; everything was improvisational.

Zak's Home Alone (2003) I started to film a "Home Alone" type video, but it didn't get far.

When I was a young child, my dad taught me how to build web pages by writing the HTML code with Microsoft Notepad. One of my early projects was getting Microsoft Agent (Merlin) to read nighttime stories (Harry Potter) to me.

With these skills, I got my first job building a website for my neighbor, Billy Al Bengston, a famous artist.

As I grew older, I started building more sophisticated pages using more powerful tools. I learned to program in the language Microsoft Visual FoxPro, an object-oriented programming language that allowed me to utilize the West-Wind Web Connect application framework. With these enterprise tools, I started to build sites that were tied to content stored in a database.

My first project was for my swim team at the time (Team Santa Monica) to allow the swimmers to track their best times, which had been done on paper to that point.

As my skill set improved, I built a site for the real-estate company Venice Properties. This site allowed the agents to login to a back office portal and upload their business information through help assisted data entry forms into the database server about each of the properties they represented. When the general public visited the site, the uploaded material was presented in a consistent format that utilized Cascading Style Sheets that were rendered from the stored content.

My dad, James Murez, is the manager of a California Certified Farmers' Market and he needed my help to automate the record keeping process. For him, I created an online tracking system used to generate Producer Certificates that list the farmers' crops.