Information Theory

• Teachers
• Weekly hours
• Competences
• Objectives
• Contents
• Activities
• Teaching methodology
• Evaluation methodology
• Bibliography
• Previous capacities

Teachers

Person in charge

• Adrián Francisco Tauste Campo ( )

Others

• Josep Vidal Manzano ( )

Weekly hours

Competences

Technical Competences

Technical competencies

• CE1 - Skillfully use mathematical concepts and methods that underlie the problems of science and data engineering.
• CE3 - Analyze complex phenomena through probability and statistics, and propose models of these types in specific situations. Formulate and solve mathematical optimization problems.
• CE7 - Demonstrate knowledge and ability to apply the necessary tools for the storage, processing and access to data.
• CE8 - Ability to choose and employ techniques of statistical modeling and data analysis, evaluating the quality of the models, validating and interpreting them.

Transversal Competences

Transversals

• CT5 [Avaluable] - Solvent use of information resources. Manage the acquisition, structuring, analysis and visualization of data and information in the field of specialty and critically evaluate the results of such management.
• CT6 - Autonomous Learning. Detect deficiencies in one's own knowledge and overcome them through critical reflection and the choice of the best action to extend this knowledge.
• CT7 - Third language. Know a third language, preferably English, with an adequate oral and written level and in line with the needs of graduates.

Basic

• CB1 - That students have demonstrated to possess and understand knowledge in an area of ??study that starts from the base of general secondary education, and is usually found at a level that, although supported by advanced textbooks, also includes some aspects that imply Knowledge from the vanguard of their field of study.
• CB2 - That the students know how to apply their knowledge to their work or vocation in a professional way and possess the skills that are usually demonstrated through the elaboration and defense of arguments and problem solving within their area of ??study.
• CB4 - That the students can transmit information, ideas, problems and solutions to a specialized and non-specialized public.
• CB5 - That the students have developed those learning skills necessary to undertake later studies with a high degree of autonomy

Generic Technical Competences

Generic

• CG5 - To be able to draw on fundamental knowledge and sound work methodologies acquired during the studies to adapt to the new technological scenarios of the future.

Objectives

1. To acquire the knowledge necessary to understand the basic principles of treatment, compression, cryptography and analysis of data based on Shannon's theory.
   Related competences: CB1, CB2, CB4, CB5, CT5, CT6, CT7, CE1, CE3, CE7, CE8, CG5,

Contents

1. Discrete random variables and processes
   Probability, ensembles of random variables, stochastic processes, Márkov processes
2. Measures of information
   Information theory, entropy, joint entropy and mutual information, data processing inequality, Fano's inequality, applications
3. Information of data sources
   Codes, asymptotic equipartition property, data compression, the high probability set, non-independent sources
4. Source coding
   Properties of codes, unique decodability, mínimum average length, Huffman codes, dictionary codes
5. Capacity of discrete channels
   Joint typical sequences, channel capacity theorem, separability of source and channel coding.
6. Channel codes
   Introduction to error correction codes, block codes
7. Cryptography
   Shannon theory of secrecy systems; main theorem; one-time pad; symmetric cryptography in practice

Activities

Activity Evaluation act

Teaching methodology

Evaluation methodology

Bibliography

Basic:

• Elements of information theory - Cover, T.M.; Thomas, J.A, John Wiley & Sons, 2006. ISBN: 0471241954
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991003402919706711&context=L&vid=34CSUC_UPC:VU1&lang=ca
• Information and communication theory - Höst, S, Wiley IEEE Press, 2019. ISBN: 9781119433781
  http://cataleg.upc.edu/record=99100491894860671~S1*cat
• Information theory, inference, and learning algorithms - Mackay, D.J.C, Cambridge University Press, 2003. ISBN: 9780521642989
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991002876809706711&context=L&vid=34CSUC_UPC:VU1&lang=ca

Previous capacities