Model Compression

In recent years, deep neural networks have shown to be highly successful for real-world applications. A large part of these successes is the increasing number of parameters of those networks, making them powerful tools for many tasks in computer vision and natural language processing. However, with an increasing number of parameters, training and inference on those neural networks becomes more and more computationally expensive. Model compression techniques mitigate this issue by removing or combining redundant parameters (model pruning) or reducing the precision of the parameters (quantisation). Model pruning techniques have shown to improve performance on downstream tasks for small compression rates — indicating interesting insights into understanding generalisation in deep learning models. This project aims to enhance understanding of the connection of model compression techniques and generalisation in deep learning.