Learning Sense-Specific Static Word Embeddings using Contextualised Word Embeddings as a Proxy

3^rd March 2021, 11:00
Yi Zhou

Abstract

Contextualised word embeddings such as BERT represent a word with a vector that considers the semantics of the target word as well its context. On the other hand, static word embeddings such as GloVe represent words by relatively low-dimensional, memory- and compute-efficient vectors but are not sensitive to the different senses of the word. In this project, we propose a method that extracts sense related information from pretrained contextualised embeddings and inject into static embeddings to create sense-specific static embeddings. Experimental results on multiple benchmarks for word sense disambiguation and sense discrimination tasks show that the proposed method can accurately learn sense-specific static embeddings, outperforming previously proposed sense-specific word embedding learning methods.

Biography

Yi Zhou is a PhD student in the Department of Computer Science at the University of Liverpool. Her research mainly focuses on Natural Language Processing. She is especially interested in how to learn semantic representations of senses/words.