High-performance backpropagation

6^th October 2021, 11:00
Navjot Kukreja

Abstract

In this talk, I will discuss my work on inverse problems - including how they are similar to training a neural network. Some issues I will discuss here are (parallel) automatic differentiation, and the enormous memory pressure of backpropagation and ways to deal with it (recomputation, lossy compression). The context of this work is Devito (https://www.devitoproject.org) - a Domain-Specific Language that started as my summer internship project and is now being utilised by the Petroleum industry on Petaflop clusters as well as on cloud platforms. Devito does just-in-time compilation to generate performance-optimised code bespoke to the target hardware.

I will also discuss my future plans involving model-parallel training of GANs for imaging applications, and other potential collaborations involving training of large neural networks under hardware constraints - on clusters, on cloud systems, or on edge devices.

Biography

I am a new Lecturer in the Department of Computer Science, with research interests around high-performance computing, domain-specific languages, inverse problems, computational imaging, machine learning, automatic differentiation, lossy compression, and various combinations th