You are here

Home » Studies » Bachelor's degrees » Bachelor Degree in Data Science and Engineering » Curriculum » Syllabus »
Credits
6
Types
Elective
Requirements
This subject has not requirements, but it has got previous capacities
Department
FIB
This course provides a technical foundation in Large Language Models (LLMs) as the core architecture of AI-driven conversational systems. The curriculum examines the principles of transformer-based architectures, pre-training methodologies, supervised and reinforcement learning-based post-training, and inference. Particular emphasis is placed on the role of scaling laws and data composition in determining model behavior. Students will analyze the application of LLMs to natural language understanding (NLU) and generation (NLG) tasks and develop methodologies to evaluate their performance.

In addition to technical capabilities, the course critically addresses the limitations and systemic risks inherent in LLMs, such as hallucinations, algorithmic bias, robustness, data privacy, and security vulnerabilities. It further explores emergent research frontiers, including computational efficiency, automated reasoning, tool-augmented generation, agentic systems, and multimodal integration. Through a synthesis of theoretical frameworks, case studies, and applied laboratory sessions, students will acquire the criteria to deploy LLMs effectively and ethically, while gaining a comprehensive perspective on the open research challenges within the field.

Teachers

Person in charge

  • Lluís Màrquez Villodre ( )

Weekly hours

Theory
2
Problems
0
Laboratory
2
Guided learning
0
Autonomous learning
6

Objectives

  1. Understanding the building blocks of modern Large Language Models, including: ML models, learning algorithms, data processing, and evaluation.
    Related competences: CG3, CT4, CE14, CE18, CE20,
  2. Be aware of the main strengths and weaknesses of LLMs, thus being critical about what one can expect from them and how to get the best of them in any situation.
    Related competences: CG2, CG3, CT4, CT5, CE18,
  3. Be aware of the principal challenges, open problems, and research directions around LLMs
    Related competences: CG2, CG8, CG9,
  4. Acquire criteria to know what kind of models/strategies can be used to address Natural Language Processing problems, being able to adapt and use an LLM for solving them.
    Related competences: CE27, CT4, CE15, CE16,
  5. Develop critical thinking about LLMs, knowing the risks associated to them, and become conscious about the necessity of using them in a fair and safe way.
    Related competences: CG8, CE16, CE20,

Contents

  1. Introduction
    In the introduction we will cover the following points: 1/ Why is this course important and necessary? 2/ Very brief history of Natural Language Processing. 3/ Brief history of Large Language Models. 4/ Where we are and where we are going.
  2. Transformers within NLP
    In this part, we will present the Transformers (self-attention) model within the context of Natural Language Processing and how this changed sequence to sequence applications:
    1/ Word representation: distributional semantics and word embeddings;
    2/ The Transformers architecture (self-attention);
    3/ Use case: seq-to-seq approach to Machine Translation.
  3. Auto-regressive Large Language Models
    This is the core part of the course, and it covers the main training steps of LLMs. More concretely, we will address: 1/ Tokenization, 2/ Pre-training, 3/ Emergent skills of LLMs: zero-shot and few-shot learning, 4/ Post training for creating conversational agents (supervised fine tuning and reinforcement learning) , 5/ Prompt engineering
  4. Limitations and Risks of LLMs
    We will discuss briefly the following topics related to the risks associated with LLMs:
    1/ Hallucinations, 2/ Bias and Fairness, 3/ LLM safety, 4/ LLM footprint, 5/ Model Collapse, 6/ LLMs and Artificial General Intelligence (AGI)
  5. Advanced topics on LLMs
    In this last part of the course we will discuss some advanced topics on LLMs, including Retrieval-Augmented Generation (RAG) and
    Training LLMs for reasoning. Time permitting, we will devote the last session to discuss also other state-of-the-art topics based on the preferences from the students.

Activities

Activity Evaluation act


Theory Classes

Hours dedicated to studying the material covered in lectures and completing the recommended readings.
Objectives: 1 2 3 4 5
Contents:
Theory
28h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
30h

Laboratory Sessions

Estimated hours dedicated to practicing the material from the laboratory sessions
Objectives: 1 2 4
Contents:
Theory
0h
Problems
0h
Laboratory
30h
Guided learning
0h
Autonomous learning
25h

Course group project

Estimated hours dedicated to carrying out the course group project (including planning, implementation, and documentation)
Objectives: 1 2 4
Contents:
Theory
1h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
25h

Individual course assignment

Horas estimadas dedicadas a realizar la práctica individual de la asignatura
Objectives: 2 5
Contents:
Theory
1h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
10h

Teaching methodology

The course introduces and explores in depth one of the most critical machine learning models for developing artificial intelligence applications today: Large Language Models (LLMs). Theoretical foundations are introduced through lectures where the instructor presents the core concepts. These lectures will also include dedicated time for discussion with students regarding previously assigned readings. These concepts are put into practice during laboratory sessions, where students learn to apply LLMs and develop solutions for specific problems. Students are required to complete and submit a final group project (2-3 people), as well as a smaller individual assignment of a more qualitative nature focused on the behavior of LLMs.

Evaluation methodology

The course is graded as follows:

F = Final exam grade
PG = Group project grade
TI = Individual work grade

Final grade = 40% F + 40% PG + 20% TI

Reassessment: Only those who have failed the final exam (an NP is not valid) may take the reassessment. The maximum grade that can be obtained in the reassessment is 7.

Evaluation of skills

The assessment of teamwork skills is based on the work carried out during the group project.

The assessment of the skills on "Solvent use of information resources" is based on the practical work (both the group project and the individual work)

Bibliography

Basic:

Previous capacities

General knowledge of machine learning and neural networks, as well as natural language processing.