Kernel Based Machine Learning and Multivariate Modeling

You are here

Credits
6
Types
Specialization complementary (Data Science)
Requirements
This subject has not requirements

Department
EIO
Kernel based Machine Learning and Multivariate Modeling

Teachers

Person in charge

  • Luis Antonio Belanche Muñoz ( )
  • Tomas Aluja Banet ( )

Weekly hours

Theory
3
Problems
0
Laboratory
0
Guided learning
0.2
Autonomous learning
6

Competences

Generic Technical Competences

Generic

  • CG3 - Capacity for mathematical modeling, calculation and experimental designing in technology and companies engineering centers, particularly in research and innovation in all areas of Computer Science.

Transversal Competences

Solvent use of the information resources

  • CTR4 - Capability to manage the acquisition, structuring, analysis and visualization of data and information in the area of informatics engineering, and critically assess the results of this effort.

Reasoning

  • CTR6 - Capacity for critical, logical and mathematical reasoning. Capability to solve problems in their area of study. Capacity for abstraction: the capability to create and use models that reflect real situations. Capability to design and implement simple experiments, and analyze and interpret their results. Capacity for analysis, synthesis and evaluation.

Technical Competences of each Specialization

Specific

  • CEC1 - Ability to apply scientific methodologies in the study and analysis of phenomena and systems in any field of Information Technology as well as in the conception, design and implementation of innovative and original computing solutions.
  • CEC3 - Ability to apply innovative solutions and make progress in the knowledge that exploit the new paradigms of Informatics, particularly in distributed environments.

Objectives

  1. Understand the foundations of Kernel-Based Learning Methods
    Related competences: CG3, CTR6,
  2. Get acquainted with specific kernel-based methods, such as the Support Vector Machine
    Related competences: CG3, CTR4,
  3. Know methods for kernelizing existing statistical or machine learning algorithms
    Related competences: CTR6,
  4. Know the theoretical foundations of kernel functions and kernel methods
    Related competences: CG3,
  5. Understanding the foundations of the Multivariate Modeling
    Related competences: CG3, CTR4,
  6. Get acquainted with multivariate modeling from latent components methods
    Related competences: CTR4, CTR6,
  7. Know modeling techniques for broad data matrices (p>n)
    Related competences: CG3, CTR4, CTR6,

Contents

  1. Introduction to Kernel-Based Learning
    This topic introduces the student the foundations of Kernel-Based Learning focusing on Kernel Linear Regression
  2. The Support Vector Machine (SVM)
    This topic develops Support Vector Machine (SVM) for classi cation, regression and novelty detection
  3. Kernels: properties & design
    This topic defines kernel functions, their properties and construction. Introduces specific kernels for different data types, such as real vectors, categorical information, feature subsets, strings, probability distributions and graphs.
  4. Kernelizing ML algorithms
    This topic reviews different techniques for kernelizing existent algorithms
  5. Theoretical underpinnings
    This topic reviews the basic theoretical underpinnings of kernel-based methods, focusing on statistical learning theory
  6. Introduction to Multivariate Modeling: Multivariate regression and Principal Component Analysis
    Extending the Regression to the multivariate case. Model formulation. Hypothesis testing. Criticism of the model
  7. Canonical Correlation Analysis
    Modeling by Canonical Correlation Analysis. Concepts and application
  8. Extending the CCA: Interbatteries Analysis and Redundancy Analysis
    Alternatives to the Canonical Correlation Analysis Modleing: The Inter-Batteries Analysis and the Redundancy Analysis. Concepts and application
  9. Modeling by Partial Least Squares
    Principal Components by a non linear iterative partial least squares (NIPALS algorithm). Partial Least Squares Regression with one dependent variable.
  10. Multivariate Partial Least Squares Regression
    PLS Regression with categorical variables, PLS-DA. Multivariate PLSR.
  11. Multi tables PLS
    Generalized Canonical Correlation Analysis. Partial Least Squares of multi tables - PLS Path Modeling

Activities

Introduction to Kernel-Based Learning

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 1
Contents:

The SVM for classi cation, regression and novelty detection

Theory
3
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 2
Contents:

Kernels: properties & design

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 1 3
Contents:

Practice class (I): the SVM

Theory
3
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 1 2
Contents:

Kernelizing ML algorithms

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 3 4
Contents:

Practice class (II): kernel design & other KBL methods

Theory
3
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 3 4
Contents:

Theoretical underpinnings

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6
Objectives: 1 4
Contents:

Introduction to Multivariate Modeling

Theory
3
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Canonical Correlation Analysis

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Extensions of the Canonical Correlation Analysis: the Inter-Batteries Analysis

Theory
3
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Extensions of CCA: Redundancy Analysis. NIPALS algorithm

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Partial Least Squares Regression 1

Theory
3
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Partial Least Squares Regression 2. PLS with categorical variables PLS - DA.

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Partial Least Squares of multi tables. PLS - PM

Theory
2
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
6

Teaching methodology

Learning is done through a combination of theoretical explanations and their application to practising exercises and real cases. The lectures will develop the necessary scientific knowledge, including its application to problem solving. These problems constitute the practical work of the students on the subject, which will be developed as autonomous learning. The software used will be primarily R.

Evaluation methodology

The course evaluation will be based on the marks obtained in the practices during the year plus the marks obtained in the written test for global evaluation.

Each practice will lead to the drafting of the corresponding written report which will be evaluated by the teachers. resulting in a mark denoted P.

The written test will be the last day of class and will evaluate the assimilation of the basic concepts on the subject, resulting in a mark denoted T.

The final mark will be obtained as:

60% x P + 40% x T

Bibliografy

Basic:

Complementary:

Web links