Topics On Optimization and Machine Learning

You are here

Credits
6
Types
Specialization compulsory (Computer Networks and Distributed Systems)
Requirements
This subject has not requirements, but it has got previous capacities
Department
AC
Mail
The goal of this course is giving the student a background in the methodologies in the advanced design of mechanisms using non-lineal convex optimization, machine learning, and deep learning that can be applied to computer networks and distributed systems.

Teachers

Person in charge

  • Jose Maria Barceló Ordinas ( )

Others

  • Jorge García Vidal ( )

Weekly hours

Theory
4
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
7

Competences

Technical Competences of each Specialization

Computer networks and distributed systems

  • CEE2.1 - Capability to understand models, problems and algorithms related to distributed systems, and to design and evaluate algorithms and systems that process the distribution problems and provide distributed services.
  • CEE2.2 - Capability to understand models, problems and algorithms related to computer networks and to design and evaluate algorithms, protocols and systems that process the complexity of computer communications networks.
  • CEE2.3 - Capability to understand models, problems and mathematical tools to analyze, design and evaluate computer networks and distributed systems.

Transversal Competences

Reasoning

  • CTR6 - Capacity for critical, logical and mathematical reasoning. Capability to solve problems in their area of study. Capacity for abstraction: the capability to create and use models that reflect real situations. Capability to design and implement simple experiments, and analyze and interpret their results. Capacity for analysis, synthesis and evaluation.

Objectives

  1. Capacity to formulate a convex optimization problem
    Related competences: CEE2.3, CTR6,
  2. Capacity to solve non linear optimization problems.
    Related competences: CEE2.3, CTR6,
  3. Capacity to apply to a real problem topics related to optimization
    Related competences: CEE2.2, CEE2.3, CEE2.1, CTR6,
  4. Capacity to understand basic machine learning algorithms
    Related competences: CEE2.3, CTR6,
  5. Capacity to apply machine learning algorithms to real scenarios.
    Related competences: CEE2.2, CEE2.3, CEE2.1, CTR6,
  6. Capacity to understand neural networks and deep learning algorithms
    Related competences: CEE2.3, CTR6,
  7. Capacity to apply neural networks and deep learning algorithms to real scenarios
    Related competences: CEE2.2, CEE2.3, CEE2.1, CTR6,

Contents

  1. Convex Optimization basics
    In this topic we will introduce the main concepts of non-linear optimization with special emphasis in convex optimization. Specifically we will see: convex sets, convex functions, convex optimization problems (COP) and duality (Lagrange dual function, KKT optimality conditions), methods for solving COP's (General Descent Methods, Interior Point Methods)
  2. Applications to machine learning topics
    Examples of how optimization is applied in the field of machine learning in computer networks and distributed networks. Specifically, we will explain supervised methods such as multiple linear regression with regularization (ridge regression and lasso), nearest neighboring methods, kernel regression (RKHS) and Gaussian processes, support vector machines, bootstrapping, random forest, and unsupervised methods such as cluttering methods with k-means, hierarchical clustering, mixture of Gaussians and the expectation maximization algorithmm.
  3. Neural networks and deep learning
    In this chapter we study the basic concepts related to neural networks and deep learning applied to computer networks and distributed systems. Specifically, introduction to neural networks, back propagation algorithm, SGD, regularization techniques and review of the most important NN architectures including multilayer perceptron (MLP), convolutional neural networks (CNN), recurrent neural networks(RNN) and autoencoders.

Activities

Activity Evaluation act


Convex Optimization basics


Objectives: 1 2 3
Contents:
Theory
20h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Applications to machine learning topics


  • Theory: Development of the theory. Embedded in the theory there will be some classes devoted to exercises.
Objectives: 4 3
Contents:
Theory
18h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Neural networks and deep learning


  • Theory: Development of the theory. Embedded in the theory there will be some classes devoted to exercises.
Objectives: 3 6
Contents:
Theory
12h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Programming project for the optimisation of a media access control protocol (MAC) in a wireless sensor network,


Objectives: 3
Theory
1h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
25h

Sensor calibration project using machine learning techniques (MLR, KNN, SVR, RF, GP),


Objectives: 5
Theory
2h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
40h

Project on neural networks and deep learning


Objectives: 7
Theory
0h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
25h

Project on programming exercises on non-linear optimization


  • Autonomous learning: Development of a project on which the student programs some non-linear optimization exercises and writes a report with the results obtained.
Objectives: 1 2
Theory
1h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
10h

Delivery of project on programming exercises on non-linear optimization


Objectives: 1 2
Week: 5
Type: assigment
Theory
0h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Delivery of the programming project for the optimisation of a media access control protocol (MAC) in a wireless sensor network,


Objectives: 2 3
Week: 9
Type: assigment
Theory
0h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Delivery of the sensor calibration project using machine learning techniques (MLR, KNN, SVR, RF, GP),


Objectives: 5 4 3
Week: 16
Type: assigment
Theory
0h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Delivery of the project using a neural network


Objectives: 3 6 7
Week: 18
Type: assigment
Theory
0h
Problems
0h
Laboratory
0h
Guided learning
0h
Autonomous learning
0h

Teaching methodology

During the initial sessions of each topic, the main results will be explained in the blackboard. The student will solve some exercises to prove their skills in the topic. Finally, the students develop projects according to the topics studied.

Evaluation methodology

The evaluation is based on the development of several projects. Each of the projects will be evaluated (0=
FM = Sum_i (Wi*Mi)

Where:

Wi = is the weight of each project i = 1, ... N
Mi = is the mark of each project i = 1, ... N

The number of projects may vary over time, but in general the following projects are foreseen:
* P1 (10%): Programming of non-linear optimisation exercises,
* P2 (25%) Programming project for the optimisation of a media access control protocol (MAC) in a wireless sensor network,
* P3 (40%): Sensor calibration project using machine learning techniques (MLR, KNN, SVR, RF, GP),
* P4 (25%): Project using a neural network

Bibliography

Basic:

Web links

Previous capacities

Recommended to have previously followed the course "Statistical Analysis of Networks and Systems (SANS-MIRI)"