BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//University of Liverpool Computer Science Seminar System//v2//EN
BEGIN:VEVENT
DTSTAMP:20260726T073824Z
UID:Seminar-dept-1055@lxserverM.csc.liv.ac.uk
ORGANIZER:CN=Lutz Oettershagen:MAILTO:Lutz.Oettershagen@liverpool.ac.uk
DTSTART:20230613T130000
DTEND:20230613T140000
SUMMARY:School Seminar Series
DESCRIPTION:Dr. Zhanxing Zhu: Theoretical understanding of learning dynamics in modern deep learning\n\nIt has been a long-standing debate that “Is deep learning alchemy or science? ”, since the success of deep learning mostly relies on various engineering design and tricks, lack of theoretical foundation. Unfortunately, the underlying mechanism of deep learning is still mysterious, which severely limits its further development from both theoretical and application aspects.\n\n\n\nIn this talk, I will introduce some of our attempts on theoretically understanding deep learning, mainly focusing on analyzing its training dynamics and tricks, including gradient descent, stochastic gradient descent (SGD), batch normalization and adversarial training.  1) We analyze the implicit regularization property of gradient descent and SGD from both local and global point of view, i.e. interpreting why SGD could find well generalizing minima compared with other alternatives; (2) We comprehensively reveal the learning dynamics of SGD with batch normalization and weight decay, named as Spherical Motion Dynamics, and show how the dynamics achieve its equilibrium state. (3) We theoretically characterize the implicit bias of adversarial training scheme that is widely used for improving the robustness resistant to adversarial attacks. Our results theoretically justify the longstanding conjecture that adversarial training modifies the decision boundary by utilizing adversarial examples to improve robustness. These new findings shed light on understanding the deep learning towards opening this black-box and also inspires new algorithmic design. Finally, a landscape for next generation of deep learning will be provided for further discussion.\n\nhttps://www.csc.liv.ac.uk/research/seminars/abstract.php?id=1055
LOCATION:Ashton Lecture Theatre
END:VEVENT
END:VCALENDAR