Project Overview:
Foundation models have revolutionized natural language processing. These are models trained on broad datasets with powerful generalization tasks such as the GPT series. There have been recent rise in vision foundation models such as GPT-4o, Instruct-BLIP and LLava. However, training foundation models has significant environmental costs tied to large scale datasets as highlighted in previous research. We propose to answer a meta question before building such models: “Can we analyze large-scale unlabelled datasets and quantify its relation to the pre-trained models’ generalization ability, without repeated training on such datasets and evaluation on downstream tasks?”. We aim to build a general purpose tool using multi-modal large language models to summarize datasets attributes without relying on annotations for these datasets, making it suitable for self supervised or vision language modelling. Such a tool can enable building foundation models with efficient cost in terms of the carbon footprint avoiding repeated training with each change in the dataset.
The project is funded through NSERC offering a postdoctoral position in the Computer Science department, University of British Columbia, Vancouver, Canada. The position is for one year with potential extension for a second year. The position is supervised by Dr. Mennatullah Siam. This project is in collaboration with our international partner RIKEN centre for Advanced Intelligence Project, Tokyo, Japan, where the postdoctoral fellow will collaborate with Dr. Naoto Yokoya and his team. We are looking for a self-motivated candidate with strong background in Computer Vision and Machine Learning.
Position Overview:
In this position, you will investigate the existing solutions for assessing datasets and developing a novel tool relying on the recent advancements in multi-modal large language models for the aforementioned goal. You will work under the supervision of Dr. Siam on the project with a generic tool that works across a multitude of applications. As part of the project you will work in collaboration with the Geoinformatics team in RIKEN centre to customize the developed tool for remote sensing applications. The goal from this project is to reduce the environmental impact of training vision foundation models through focusing on studying the broad datasets used standalone before training such large-scale models.
As a Postdoctoral Fellow, you will:
- Perform literature review of related works on the aforementioned topic.
- Implement and develop dataset assessment tool that is sufficiently broad to work across various applications.
- Analyze and build datasets to develop better vision foundation models that are trained on them with an efficient training cost.
- Collaborate with researchers from other disciplines, specifically researchers focused on remote sensing to study the use of this tool in this application
- Lead and mentor visiting students as part of an interdisciplinary team
- Contribute to publishing research findings in top Computer Vision conferences, CVPR, ICCV, ECCV and peer reviewed journals.
Minimum Qualifications:
- PhD in Computer Science or a related field obtained within the last five years.
- Strong skills in machine learning and deep learning.
- Strong Computer Vision background with publications in top/mid-tier conferences e.g., CVPR, ICCV, ECCV, WACV.
- Strong mathematical understanding and skills in applying Machine Learning/Deep Learning solutions.
- Proficiency in Python programming and PyTorch library.
Offer:
- Full-time position (1 year, with the potential to extend to a second year)
- Benefits: UBC offers a comprehensive benefits plan, including extended health and dental coverage
We strive to create a respectful, positive and safe working environment for people of all backgrounds. We believe that inclusiveness and diversity are essential to academic excellence. We encourage members of underrepresented groups to apply including First Nations, Métis and Inuit peoples, Indigenous peoples of North America, Black-identified persons, other racialized persons, persons with disabilities, and those who identify as women and/or 2SLGBTQ+. Priority will be given to permanent residents or Canadian citizens.
Applications will be reviewed till the position is filled. For inquiries, contact Dr. Mennatullah Siam (mennatullah.siam@ubc.ca).
How to apply?
Please send applications to mennatullah.siam@ubc.ca and refer to reference number PDFO-58478.
Desired start date: 01 Aug 2025
Duration: Fixed term / Temporary
Contract Type: Full Time
Equity and diversity are essential to academic excellence. An open and diverse community fosters the inclusion of voices that have been underrepresented or discouraged. We encourage applications from members of groups that have been marginalized on any grounds enumerated under the B.C. Human Rights Code, including sex, sexual orientation, gender identity or expression, racialization, disability, political belief, religion, marital or family status, age, and/or status as a First Nation, Metis, Inuit, or Indigenous person.
