Pruning Deep Neural Networks with Multi-Arm Bandits

6^th November 2019, 11:00
Prof Sunil Vadera
University of Salford

Abstract

Advances in deep learning have started a new AI revolution that is transforming our world. Applications in speech recognition, self-driving cars and publicity around systems such Alpha Go defeating human Go champions has ignited interest from the public, academia and industry.
Convolutional neural networks, that can take images as input, learn to identify key features and perform classification are the heart of many of the proposed applications in medical diagnosis such as detecting breast cancer, predicting Alzheimer’s disease and grading brain tumours. These neural networks can, however, be very large, taking up memory and requiring significant computational resources. For example, one of the most highly cited networks, AlexNet has over 62 million parameters that need to be learned.

This seminar explores methods for reducing the size of such networks without compromising performance. The talk will summarise existing methods for pruning neural networks, including direct methods, and Optimal Brain Damage, and describe our recent work on a new framework, based on the use of multi-armed bandits such as Thompson Sampling and Upper Confidence Bounds.

The talk will conclude with the results of an empirical evaluation of the new methods over several benchmark data sets such as ImageNet, MNIST, CIFAR and Street View House Numbers.