PhD project in Shrinking the carbon footprint of deep learning with dynamical systems

Deep learning (DL) has led to reaching notably higher levels of artificial intelligence (AI) over the past decade. The speed of growth of DL has no historical precedent, disrupting every single industry. However, all those advances require computationally heavy operations which are power-hungry. Consequently, DL has a terrible carbon footprint, found to be comparable to that of its aviation and automotive fossil-fuel counterparts. There is a high risk of DL becoming a significant contributor to speeding up the catastrophic climate change. There is a great urgency to curb the currently exorbitant power demands of DL. This could be achieved by enhancing its energy efficiency. DL’s energy efficiency stretches over algorithms, architectures, circuits, devices and cooling technology.

This project will focus on achieving energy efficiency improvements at algorithmic level. The main objective of this research is to make DL more resource-efficient and thus less environmentally damaging by looking at DL through the lens of dynamical systems (DSs). We will leverage the rich theory and well-established numerical techniques of DSs to develop novel and theoretically sound tools towards: (i) shortening the duration of the notoriously long training of Transformers, which is a popular DL model that has recently seen some staggering success in many domains; (ii) accelerating the discovery of DL architectures that achieve the best possible performance for computer vision tasks. The latter task is critical for DL and also known to be suffering from a prohibitively high computational cost required to handle the vast search space.

The efficiency of the developed tools will be validated on benchmark datasets and tasks. Both publicly available datasets and datasets provided by Prof Amin Al-Habaibeh’s group will be analysed.

The proposed cutting-edge research will be carried out at the Department of Physics and Mathematics, School of Science and Technology, Nottingham Trent University (NTU), under the guidance of the supervisory team (Dr Archontis Giannakidis, Dr Jonathan Crofts, Prof Amin Al-Habaibeh).

The successful candidate will work in a highly stimulating research environment. In REF 2021, 83% of NTU’s research activity was classified either as “world-leading” or “internationally excellent”

As part of their role, the student will have the opportunity to publish their research at top-tier machine learning conferences (such as ICML, ICLR, NIPS etc.).