Advanced Topics in Computational Intelligence

Credits
4
Department
CS
Types
Elective
Requirements
This subject has not requirements
The aim of this course is to present to the students, different advanced techniques in computational intelligence. Once acquired the basic knowledge of fuzzy, evolutionary and neural computation in the CI-MAI course, the students are ready to go through more interesting and powerful computational intelligence approaches such are hybrid techniques: neuro-fuzzy and genetic-fuzzy systems, fuzzy inductive reasoning, fuzzy and heterogeneous neural networks and neural networks trained by means of evolutionary algorithms, as well as recurrent neural networks and incremental methods for neural networks construction.

Teachers

Person in charge

  • Maria Angela Nebot Castells ( )

Others

  • Enrique Romero Merino ( )
  • Luis Antonio Belanche Muñoz ( )
  • René Alquezar Mancho ( )

Weekly hours

Theory
1.8
Problems
0
Laboratory
0.9
Guided learning
0
Autonomous learning
5

Competences

Generic Technical Competences

Generic

  • CG3 - Capacity for modeling, calculation, simulation, development and implementation in technology and company engineering centers, particularly in research, development and innovation in all areas related to Artificial Intelligence.
  • CG4 - Capacity for general management, technical management and research projects management, development and innovation in companies and technology centers in the area of Artificial Intelligence.

Technical Competences of each Specialization

Academic

  • CEA11 - Capability to understand the advanced techniques of Computational Intelligence, and to know how to design, implement and apply these techniques in the development of intelligent applications, services or systems.

Professional

  • CEP2 - Capability to solve the decision making problems from different organizations, integrating intelligent tools.
  • CEP3 - Capacity for applying Artificial Intelligence techniques in technological and industrial environments to improve quality and productivity.

Transversal Competences

Teamwork

  • CT3 - Ability to work as a member of an interdisciplinary team, as a normal member or performing direction tasks, in order to develop projects with pragmatism and sense of responsibility, making commitments taking into account the available resources.

Objectives

  1. To understand the fuzzy inductive reasoning methodology for modelling systems and predicting their behavior.
    Related competences: CEA11, CG3, CG4,
  2. To apply the fuzzy inductive reasoning methodology to the simulation of environmental, biomedical, industrial or economical processes.
    Related competences: CT3, CEP2, CEP3,
  3. To understand the different ways of designing computational intelligence hybrid techniques by integrating
    fuzzy logic, neural networks and evolutionary algorithms.
    Related competences: CEA11, CG3, CG4,
  4. To apply computational intelligence hybrid techniques to solve complex data mining problems in real scenarios.
    Related competences: CT3, CEP2, CEP3,
  5. To understand some of the most advanced and recent techniques in the field of neural networks (e.g. recurrent neural nets, extreme learning machines, deep neural nets).
    Related competences: CEA11, CG3, CG4,
  6. To apply neural network advanced techniques to solve complex data mining problems in real scenarios.
    Related competences: CT3, CEP2, CEP3,

Contents

  1. Fuzzy inductive reasoning
    The fuzzy inductive reasoning (FIR) methodology allows the qualitative modelling of systems and the quantitative prediction of their behavior.
  2. Hybrid fuzzy systems: neuro-fuzzy systems and genetic-fuzzy systems
    The hybrid fuzzy systems improve the abilities of fuzzy systems by introducing neural networks and genetic algorithms to learn and adapt their parameters for a better performance.
  3. Fuzzy and heterogeneous neural networks
    Similarity-based neural networks, possibly trained using evolutionary algorithms, allow the processing of fuzzy and heterogeneous data in classification or regression problems without the need of data coding.
  4. Recurrent neural networks
    Discrete-time recurrent neural networks allow the learning and processing of dynamic input/output tasks such as time-series prediction, sequence classification and translation.
  5. Incremental methods for neural network construction and extreme learning machines
    Incremental methods for neural network construction allow an efficient computation of simple models with good generalization performance. Extreme learning machines do so by assigning random weights to some part of the neural architecture and optimizing the rest of weights.
  6. Deep neural networks
    Deep neural networks ,,,, (to complete)

Activities

Fuzzy Inductive Reasoning

Development of the corresponding topic and laboratory exercises
Theory
1.8
Problems
0
Laboratory
2
Guided learning
0
Autonomous learning
4
Objectives: 1 2
Contents:

Hybrid fuzzy systems

Development of the corresponding topic and laboratory exercises
Theory
3.6
Problems
0
Laboratory
2
Guided learning
0
Autonomous learning
6
Objectives: 3 4
Contents:

Fuzzy and heterogeneous neural networks

Development of the corresponding topic
Theory
3.6
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
4
Objectives: 3
Contents:

Recurrent neural networks

Development of the corresponding topic and laboratory exercises
Theory
4.5
Problems
0
Laboratory
4
Guided learning
0
Autonomous learning
9
Objectives: 5 6
Contents:

Incremental neural networks and extreme learning machines

Development of the corresponding topic and laboratory exercises
Theory
2.7
Problems
0
Laboratory
2
Guided learning
0
Autonomous learning
5
Objectives: 5 6
Contents:

Deep neural networks

Development of the corresponding topic and laboratory exercises
Theory
5.4
Problems
0
Laboratory
2
Guided learning
0
Autonomous learning
8
Objectives: 5 6
Contents:

Teaching methodology

Theory classes will introduce the knowledge, techniques and concepts required to apply them
in practice during the laboratory classes. Theory classes will be mainly of the type magisterial
lecture, but some of them may be of the type exposition-participation, with the participation of
the students in solving problems or exercises.

Laboratory classes have as objective that the students work with software tools which allow the
application to real problems of the techniques presented in theory classes. Students will use
these tools to develop their practical work of the course, which will consist of a part of autonomous
individual work and a part of cooperative work in a team of 2/3 people. Some time of the laboratory
classes will be devoted to the orientation and supervision by the professor of these autonomous
and cooperative works.

Once each two weeks, there will be a laboratory session of two hours. Previously to this session,
there will have been two theory sessions (one for week) with a total of four hours. Depending on
logistic matters, these two theory sessions might be of two hours each or one of three hours
(the week without laboratory class) and the other one of one hour (immediately before the
laboratory session).

A final exam will evaluate the specific objectives of understanding the concepts and methods
presented during the course. On the other hand, the individual and cooperative practical works
by the students will allow the evaluation of the specific objectives of applying the presented
techniques, as well as the general, basic and transverse skills associated with the course.

Evaluation methodology

The technical skills mark (M) is calculated as follows:

M = 0.40*FINAL_EXAM + 0.60*PRACT_WORK

where

* FINAL_EXAM refers to the mark of the final exam;
* PRACT_WORK refers to the global mark of the practical works defined by the teachers during the course and carried out in small groups.

Additionally, interested students can do a voluntary extra work, defined by them and approved by the teachers. This work will give to the student a maximum of 2 extra points in the final mark, M.

Nevertheless, M will be NP if the student do no present anything in the assessment activities.

The generic skill (Teamwork) mark will be directly given by PRACT_WORK and the voluntary extra work (if it has been done).

Bibliografy

Basic:

  • Fuzzy inductive reasoning for variable selection analysis and modelling of biological systems - NEBOT, Àngela; CELLIER, François E; CARVAJAL, Raúl; MUGICA, Francisco, International Journal of General Systems , 2009 / Volume 38, Issue 8, pages 793-811. ISBN: 0308-1079
  • Foundations of Neuro-Fuzzy Systems - NAUCK, Detlef; KLAWONN, Frank; KRUSE, Rudolf, John Wiley & sons , 1997. ISBN: 978-0471971511
    http://cataleg.upc.edu/record=b1132739~S1*cat
  • Genetic Fuzzy Systems: Evolutionary Tuning and Learning of Fuzzy Knowledge Bases - CORDÓN, Oscar; HERRERA, Francisco; HOFFMANN, Frank; MAGDALENA, Luis , World Scientific , 2001. ISBN: 978-981-02-4016-5
  • Fuzzy Neural Network Theory and Application - LIU, Puyin; LI, Hongxing , World Scientific , 2004. ISBN: 978-981-238-786-8
  • Recurrent neural networks - SCHMIDHUBER, Jürgen, , updated 2011. ISBN:
    http://www.idsia.ch/~juergen/rnn.html
  • Extreme learning machine: Theory and applications - HUANG, Guang-Bin; ZHU, Qin-Yu; SIEW, Chee-Kheong, Neurocomputing , 2006 / Vol. 70, pp 489–501. ISBN: 0925-2312
  • A fast learning algorithm for deep belief nets - HINTON, G. E; OSINDERO, S.; and TEH, Y. W. , Neural Computation , 2006 / Vol. 18, pp:1527-1554. ISBN: 0899-7667
  • Reducing the dimensionality of data with neural networks - HINTON, G. E. and SALAKHUTDINOV, R. R. , Science , 2006 / Vol. 313, pp:504-507. ISBN: 0036-8075

Complementary:

  • Confidence measures for predictions in fuzzy inductive reasoning - CELLIER, François E; LÓPEZ, Josefina; NEBOT, Àngela; CEMBRANO, Gabriela, International Journal of General Systems , 2010 / Vol.39 (8), pp: 839-853. ISBN: 0308-1079
  • Optimization of fuzzy partitions for inductive reasoning using genetic algorithms - ACOSTA, Jesús.; NEBOT, Àngela; VILLAR, Pedro; and FUERTES, Josep Maria, International Journal of Systems Science , 2007 / Vol. 38, No. 12, pp: 991–1011. ISBN: 0020-7721
  • Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence - JANG, Jyh-Shing Roger; SUN, Chuen-Tsai; MIZUTANI, Eiji , Prentice Hall , 1997. ISBN: 978-0132610667
  • Heterogeneous Neural Networks: Theory and Applications - BELANCHE, Lluís, Universitat Politècnica de Catalunya, Ph.D. thesis , June 2000. ISBN:
    http://www.lsi.upc.edu/~belanche/PhDThesis/TLABM1de3.pdf
  • Discrete-time recurrent neural networks - FORCADA, Mikel L., , 2002. ISBN:
    http://www.dlsi.ua.es/~mlf/nnafmc/pbook/node21.html
  • Recurrent Neural Networks - HU, Xiaolin; and BALASUBRAMANIAM P. (eds.), InTech , 2008. ISBN: 978-953-7619-08-4
  • Constructive Neural Networks - FRANCO, Leonardo; JEREZ, José M. (Eds.), Springer , 2010. ISBN: 978-3-642-04512-7
  • Extreme learning machines: a survey - HUANG, Guang-Bin; WANG, Dian Hui; LAN, Yuan, International Journal of Machine Learning and Cybernetics , 2011 / Vol. 2, pp:107–122. ISBN: 1868-8071
  • Exploring Strategies for Training Deep Neural Networks - LAROCHELLE, Hugo; BENGIO, Yoshua; LOURADOUR, Jerôme; LAMBLIN, Pascal , Journal of Machine Learning Research , 2009 / Vol.1, pp:1-40 . ISBN: 1533-7928

Web links

Previous capacities

Basic concepts of Computational Intelligence, mainly neural networks, fuzzy logic and evolutionary algorithms