Topics On Optimization and Machine Learning

• Teachers
• Weekly hours
• Competences
• Objectives
• Contents
• Activities
• Teaching methodology
• Evaluation methodology
• Bibliography
• Web links
• Previous capacities

Teachers

Person in charge

• Jose Maria Barceló Ordinas ( )

Others

• Jorge García Vidal ( )

Weekly hours

Competences

Technical Competences of each Specialization

Computer networks and distributed systems

• CEE2.1 - Capability to understand models, problems and algorithms related to distributed systems, and to design and evaluate algorithms and systems that process the distribution problems and provide distributed services.
• CEE2.2 - Capability to understand models, problems and algorithms related to computer networks and to design and evaluate algorithms, protocols and systems that process the complexity of computer communications networks.
• CEE2.3 - Capability to understand models, problems and mathematical tools to analyze, design and evaluate computer networks and distributed systems.

Transversal Competences

Reasoning

• CTR6 - Capacity for critical, logical and mathematical reasoning. Capability to solve problems in their area of study. Capacity for abstraction: the capability to create and use models that reflect real situations. Capability to design and implement simple experiments, and analyze and interpret their results. Capacity for analysis, synthesis and evaluation.

Objectives

1. Capacity to formulate a convex optimization problem
   Related competences: CEE2.3, CTR6,
2. Capacity to solve non linear optimization problems.
   Related competences: CEE2.3, CTR6,
3. Capacity to apply to a real problem topics related to optimization
   Related competences: CEE2.2, CEE2.3, CEE2.1, CTR6,
4. Capacity to understand basic machine learning algorithms
   Related competences: CEE2.3, CTR6,
5. Capacity to apply machine learning algorithms to real scenarios.
   Related competences: CEE2.2, CEE2.3, CEE2.1, CTR6,
6. Capacity to understand neural networks and deep learning algorithms
   Related competences: CEE2.3, CTR6,
7. Capacity to apply neural networks and deep learning algorithms to real scenarios
   Related competences: CEE2.2, CEE2.3, CEE2.1, CTR6,

Contents

1. Convex Optimization basics
   In this topic we will introduce the main concepts of non-linear optimization with special emphasis in convex optimization. Specifically we will see: convex sets, convex functions, convex optimization problems (COP) and duality (Lagrange dual function, KKT optimality conditions), methods for solving COP's (General Descent Methods, Interior Point Methods)
2. Applications to machine learning topics
   Examples of how optimization is applied in the field of machine learning in computer networks and distributed networks. Specifically, we will explain supervised methods such as multiple linear regression with regularization (ridge regression and lasso), nearest neighboring methods, kernel regression (RKHS) and Gaussian processes, support vector machines, bootstrapping, random forest, and unsupervised methods such as cluttering methods with k-means, hierarchical clustering, mixture of Gaussians and the expectation maximization algorithmm.
3. Neural networks and deep learning
   In this chapter we study the basic concepts related to neural networks and deep learning applied to computer networks and distributed systems. Specifically, introduction to neural networks, back propagation algorithm, SGD, regularization techniques and review of the most important NN architectures including multilayer perceptron (MLP), convolutional neural networks (CNN), recurrent neural networks(RNN) and autoencoders.

Activities

Activity Evaluation act

Teaching methodology

Evaluation methodology

Bibliography

Basic:

• Convex optimization - Boyd, S.P.; Vandenberghe, L, Cambridge University Press, 2004. ISBN: 0521833787
  http://cataleg.upc.edu/record=b1253105~S1*cat
• Pattern recognition and machine learning - Bishop, Christopher M, Springer, 2006. ISBN: 0387310738
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991003157379706711&context=L&vid=34CSUC_UPC:VU1&lang=ca
• Machine learning : a bayesian and optimization perspective - Theodoridis, S, Elsevier Academic Press, 2020. ISBN: 9780128188033
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991004192339706711&context=L&vid=34CSUC_UPC:VU1&lang=ca
• Deep learning - Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron, The MIT Press, [2016]. ISBN: 9780262035613
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991004107709706711&context=L&vid=34CSUC_UPC:VU1&lang=ca

Web links

• Book in PDF: "Convex Optimization" of Stephen P. Boyd and Lieven Vandenberghe, http://www.stanford.edu/~boyd/cvxbook/
• Book in PDF and supplementary material for "Deep Learning" of Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron https://www.deeplearningbook.org

Previous capacities