Project details
The aim of this Ph.D. project is to develop a comprehensive framework that seamlessly integrates Large Language Models (LLMs) with advanced computer vision techniques to enhance the understanding, generation, and retrieval of multimedia content.
Example Research Directions:
1. Multimodal Representation Learning: Investigate methods for joint representation learning that fuses textual and visual information. This could involve adapting pre-trained LLMs to handle multimodal input, and training models to map between the semantic space of language and the visual space.
2. Content Generation: Explore how LLMs can be used to generate descriptive and contextually relevant text for images and videos. This could involve conditioning the language model on visual features extracted by a convolutional neural network (CNN).
3. Semantic Understanding: Explore how the combined model can achieve a deeper semantic understanding of content. This may involve incorporating domain-specific knowledge into the LLM or fine-tuning the model on specific datasets.
Outcomes and Impact: The developed framework and algorithms will be validated in real-world applications, such as healthcare, sports, robotics, content creation platforms, or assistive technologies. The model and learning performance will be evaluated in terms of accuracy, robustness, and user subjective satisfaction. The scalability and efficiency of the proposed multimodal approach will be evaluated, allowing its potential it to handle large datasets and operate in real-time systems. The project will evaluate the model’s performance on benchmark datasets and its application potential in collaboration with industry partners.
Expected Contributions: The project will contribute to the field by providing a unified approach for combining LLMs and computer vision to enhance multimodal understanding, and address the challenges of representation learning, cognition tasks, and information retrieval in a holistic manner, with potential applications in science, engineering and various industries.
The Department of Computer Science has an excellent research record in AI, machine learning, robotics, computer vision and data science. The successful candidate will have access to robotics and AI laboratories, high-spec computing facilities (e.g., GPUs), HPC, and £5.8M DigLabs, complementing a £9m investment in research and teaching. You will have regular supervision meetings and work with a strong AI research team (over 30 PhDs/PDRAs/academic staff) at Loughborough and opportunities to work with our collaborators in industry.
Entry requirements
Our entry requirements are listed using standard UK undergraduate degree classifications i.e. first-class honours, upper second-class honours and lower second-class honours. To learn the equivalent for your country, please choose it from the drop-down below.
Entry requirements for United Kingdom
Applicants should have, or expect to achieve, at least a 2:1 Honours degree in computer science or a related science and engineering subject. A relevant Master’s degree, experience in machine learning, deep learning and computer vision, strong programming skills, and passion in interdisciplinary research and innovation will be advantages.
English language requirements
Applicants must meet the minimum English language requirements. Further details are available on the International website.
Fees and funding
Tuition fees cover the cost of your teaching, assessment and operating University facilities such as the library, IT equipment and other support services. University fees and charges can be paid in advance and there are several methods of payment, including online payments and payment by instalment. Fees are reviewed annually and are likely to increase to take into account inflationary pressures.
How to apply
All applications should be made online. Under programme name, select Computer Science. Please quote the advertised reference number: CO/BL-Un1/2024 in your application.
To avoid delays in processing your application, please ensure that you submit a CV and the minimum supporting documents.
The following selection criteria used by the academic School to help them make a decision on your application, will be the same as we use for funded studentships.
Please note, applications for this project are considered on an ongoing basis once submitted and the project may be withdrawn prior to the application deadline, if a suitable candidate is chosen for the project.
We support excellent applicants from China to apply for China Scholarship Council (CSC) funding for PhD projects starting from October 2024. Applications would need to be made by 31st January 2024, to be eligible for funding. Further details on how to apply and associated application deadlines are available on our Research degree funding webpage.
