The Pick-to-Learn Algorithm: Self-certifying ML

14^th November 2023, 13:00 Ashton Lecture Theatre
Dr. Dario Paccagnan
Department of Computing, Imperial College London

Abstract

Generalization bounds are valuable both for theory and applications. On the one hand, they shed light on the mechanisms that underpin the learning processes; on the other, they certify how well a learned model performs against unseen inputs. In this work we build upon a recent breakthrough in compression theory to develop a new framework yielding tight generalization bounds of wide practical applicability. The core idea is to embed any given learning algorithm into a suitably-constructed meta-algorithm (here called Pick-to-Learn, P2L) in order to instill desirable compression properties. When applied to the MNIST classification dataset and to a synthetic regression problem, P2L not only attains generalization bounds that compare favorably with the state of the art (test-set and PAC-Bayes bounds), but it also learns models with better post-training performance.

Paper accepted as Spotlight at NeurIPS 2023.
Joint work with Marco C. Campi and Simone Garatti

Biography

Dario Paccagnan is a Senior Lecturer at the Department of Computing, Imperial College London. Before joining Imperial, he was a postdoctoral fellow with the Center for Control, Dynamical Systems and Computation, University of California, Santa Barbara. He obtained his PhD from the Automatic Control Laboratory, ETH Zurich, Switzerland, in 2018. He received a B.Sc. and M.Sc. in Aerospace Engineering from the University of Padova, Italy, in 2011 and 2014, and a M.Sc. in Mathematical Modelling and Computation from the Technical University of Denmark in 2014; all with Honors. Dario’s interests are at the interface of game theory, optimization, and control theory. Dario was recognized with the ETH medal for his doctoral work, the SNSF Early Postdoc Mobility Fellowship, and the SNSF Doc Mobility Fellowship.

Additional Materials

• https://openreview.net/forum?id=40L3viVWQN