Ofertes de projectes

State-of-the-art models such as LLMs are too large to fit in a single compute node (GPU, NPU, CPU), both for training and inference on a device (e.g., phone, laptop, tablet) or in larger-scale data centers. There is a need to develop optimization techniques to split and place these models onto a distributed set of compute nodes so that the overall system performance is maximized. The research will be focused on optimizing the placement of AI models onto distributed systems considering training time, energy consumption, and computational resources. 

In this thesis, the student will model and simulate distributed AI workloads using both mathematical frameworks and simulators. This encompasses the modeling of network, compute, and memory components within a distributed architecture. Developing new modules to enhance the modeling process. Evaluating and optimizing various parallelization techniques to improve overall system performance.

The Universitat Politècnica de Catalunya · BarcelonaTech offers Master thesis fellowships in the field of LLM training. The research will be supported by Qualcomm and will be carried out in an environment with a strong interaction with leading experts in the field, with opportunities for doing internships in the company.

More information here: https://www.cs.upc.edu/~jordicf/priv/eda/llm_qc.html

Automatic gaze tracking methods have been used in the recent years for various purposes, including measuring ad attention or enabling for gesture-less user-device interactions. Most of these methods have been tested in the desktop setting, while the few reported attempts that addressed the mobile setting report low accuracy and require continuous calibration. Additionally, the aforementioned methods rely on expensive sensors, such as infrared eye trackers, that limit their scaling capacity. Other methods employ touch-based interactions (e.g., tracking zoom/pinch gestures and scroll activity) to produce an estimation of the user gaze which, at best, are weakly correlated with visual attention. However, 42% of the time spent on websites is by mobile users, while similar trends are reported for the percentage of page views per visit, which creates an opportunity for a novel attention measurement technology to take root; one which can offer accurate, reliable, and scalable measurements.

In the proposed project, we are interested in the mobile setting, and propose the use of depth information, on top of the usual RGB (Red, Green, Blue) pixel data acquired by mobile device cameras, to track and quantify visual attention. Depth information can either be estimated with post computer visual processing or acquired directly with high precision, using advanced LiDAR sensors that newer smartphone models are currently equipped with. Furthermore, we propose the use of "lightweight" Deep Learning (DL) techniques to achieve an accurate and robust gaze point estimation without the need of extensive calibration. Unlike prior art, we will consider the use of historical information to inform future predictions and will examine how such models can be tailored to each user by injecting personalized data. In addition, we propose a more lightweight Neural Network (NN) architecture, suitable for the processing power of mobile devices, that also leverages the use of video or photo frames with depth sensory data.

The candidate will:

• Develop an app for iPhone (in Swift) for collecting gaze information and image data (using the device's frontal camera and LiDAR sensors), while completing a simple task of tracking a red dot in a matrix.
• Incorporate in the app the functionality offered by SensingKit,  an open-source mobile sensing framework compatible with iOS specialised in capturing motion, orientation, location and proximity (using iBeacon¿ or Eddystone¿) data and transmitting them on a server over any Internet connection. [https://github.com/SensingKit/SensingKit-iOS]
• Carry out a crowdsourcing study and collect data from a number of participants.
• Train lightweight DL models to predict accurately gaze point estimation, using the collected data, and compare against prior baselines/experiments (these should be replicated).
• Carry out ablation studies, personalisation techniques, error analyses, etc.
• Contribute to authoring a scientific article