Kernel-Based Machine Learning and Multivariate Modelling

You are here

Credits
6
Types
Specialization complementary (Data Science)
Requirements
This subject has not requirements
Department
EIO
Kernel based Machine Learning and Multivariate Modeling

Teachers

Person in charge

  • Luis Antonio Belanche Muñoz ( )

Others

  • Marta Janira Castellano Palomino ( )
  • Tomas Aluja Banet ( )

Weekly hours

Theory
3
Problems
0
Laboratory
0
Guided learning
0.2
Autonomous learning
6

Competences

Generic Technical Competences

Generic

  • CG3 - Capacity for mathematical modeling, calculation and experimental designing in technology and companies engineering centers, particularly in research and innovation in all areas of Computer Science.

Transversal Competences

Information literacy

  • CTR4 - Capability to manage the acquisition, structuring, analysis and visualization of data and information in the area of informatics engineering, and critically assess the results of this effort.

Reasoning

  • CTR6 - Capacity for critical, logical and mathematical reasoning. Capability to solve problems in their area of study. Capacity for abstraction: the capability to create and use models that reflect real situations. Capability to design and implement simple experiments, and analyze and interpret their results. Capacity for analysis, synthesis and evaluation.

Technical Competences of each Specialization

Specific

  • CEC1 - Ability to apply scientific methodologies in the study and analysis of phenomena and systems in any field of Information Technology as well as in the conception, design and implementation of innovative and original computing solutions.
  • CEC3 - Ability to apply innovative solutions and make progress in the knowledge that exploit the new paradigms of Informatics, particularly in distributed environments.

Objectives

  1. Understand the foundations of Kernel-Based Learning Methods
    Related competences: CG3, CTR6,
  2. Get acquainted with specific kernel-based methods, such as the Support Vector Machine
    Related competences: CG3, CTR4,
  3. Know methods for kernelizing existing statistical or machine learning algorithms
    Related competences: CTR6,
  4. Know the theoretical foundations of kernel functions and kernel methods
    Related competences: CG3,
  5. Understanding the foundations of the Multivariate Modeling
    Related competences: CG3, CTR4,
  6. Get acquainted with multivariate modeling from latent components methods
    Related competences: CTR4, CTR6,
  7. Know modeling techniques for broad data matrices (p>n)
    Related competences: CG3, CTR4, CTR6,

Contents

  1. Introduction to Kernel-Based Learning
    This topic introduces the student the foundations of Kernel-Based Learning focusing on Kernel Linear Regression
  2. The Support Vector Machine (SVM)
    This topic develops Support Vector Machine (SVM) for classification, regression and novelty detection
  3. Kernels: properties & design
    This topic defines kernel functions, their properties and construction. Introduces specific kernels for different data types, such as real vectors, categorical information, feature subsets, strings, probability distributions and graphs.
  4. Kernelizing ML algorithms
    This topic reviews different techniques for kernelizing existent algorithms
  5. Theoretical underpinnings
    This topic reviews the basic theoretical underpinnings of kernel-based methods, focusing on statistical learning theory
  6. Introduction to Multivariate Modeling: Principal Component Regression
    Extending the Regression to the multivariate case. Principal Component Analysis and Regression
  7. Partial Least Squares Regression
    Modeling by PLSR1. Algorithm and properties. The multivariate case PLSR2
  8. Foundations for advanced neural networks
    Hierarchical complexity, shallow and deep neural networks, logistic regression with neural networks.
  9. Deep Learning for supervised learning
    Deep neural networks, convolutional neural networks.
  10. Deep Learning for unsupervised learning
    Autoencoders, deep belief networks.
  11. Recurrent neural networks
    Sequence models, reservoir computing, long-short term memory

Activities

Activity Evaluation act


Introduction to Kernel-Based Learning


Objectives: 1
Contents:
Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

The SVM for classification, regression and novelty detection


Objectives: 2
Contents:
Theory
3h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Kernels: properties & design


Objectives: 1 3
Contents:
Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Practice class (I): the SVM


Objectives: 1 2
Contents:
Theory
3h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Practice class (II): kernel design & other KBL methods


Objectives: 3 4
Contents:
Theory
3h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Theoretical underpinnings


Objectives: 1 4
Contents:
Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Introduction to Multivariate Modeling. Principal Component Regression.



Theory
3h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Partial Least Squares Regression 1.



Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Partial Least Squares Regression 2.



Theory
3h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Foundations for advanced neural networks



Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Deep Learning for supervised learning



Theory
3h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Deep Learning for unsupervised learning



Theory
4h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
6h

Recurrent neural networks



Theory
2h
Problems
0h
Laboratory
0h
Guided learning
0.2h
Autonomous learning
6h

Evaluation quiz


Objectives: 1 2 3 4 5 6 7
Week: 15 (Outside class hours)
Type: final exam
Theory
0h
Problems
0h
Laboratory
0h
Guided learning
3h
Autonomous learning
12h

Teaching methodology

Learning is done through a combination of theoretical explanations and their application to practising exercises and real cases. The lectures will develop the necessary scientific knowledge, including its application to problem solving. These problems constitute the practical work of the students on the subject, which will be developed as autonomous learning. The software used will be primarily R.

Evaluation methodology

The course evaluation will be based on the marks obtained in the practices during the year plus the marks obtained in the written test for global evaluation.

Each practice will lead to the drafting of the corresponding written report which will be evaluated by the teachers. resulting in a mark denoted P.

The written test will be the last day of class and will evaluate the assimilation of the basic concepts on the subject, resulting in a mark denoted T.

The final mark will be obtained as:

60% x P + 40% x T

Bibliography

Basic:

Complementary:

Web links