This course will train students in the competencies necessary to design and configure analytical databases, evaluating the different possible alternatives in the context of their company. Concepts of generic relational databases (applicable to decision-making environments for data storage) are dealt with, in order to further study non-relational alternatives, also known as NOSQL managers, most appropriate for Big Data environments. First, data warehouse concepts and then key-value and columnars and architectures (distributed and in memory) will be presented alternatives to relational databases in certain scenarios. An introduction to the design and configuration of non-relational databases is also included, emphasizing the configuration in analytical environments and, including massive data processing in functional style environments. The knowledge given is essential to face the tasks of the data engineer.
Person in charge
Alberto Abello Gamazo (
Sergi Nadal Francesch (
CE7 - Demonstrate knowledge and ability to apply the necessary tools for the storage, processing and access to data.
CT4 - Teamwork. Be able to work as a member of an interdisciplinary team, either as a member or conducting management tasks, with the aim of contributing to develop projects with pragmatism and a sense of responsibility, taking commitments taking into account available resources.
CT6 [Avaluable] - Autonomous Learning. Detect deficiencies in one's own knowledge and overcome them through critical reflection and the choice of the best action to extend this knowledge.
CB2 - That the students know how to apply their knowledge to their work or vocation in a professional way and possess the skills that are usually demonstrated through the elaboration and defense of arguments and problem solving within their area of ??study.
CB3 - That students have the ability to gather and interpret relevant data (usually within their area of ??study) to make judgments that include a reflection on relevant social, scientific or ethical issues.
Generic Technical Competences
CG1 - To design computer systems that integrate data of provenances and very diverse forms, create with them mathematical models, reason on these models and act accordingly, learning from experience.
CG2 - Choose and apply the most appropriate methods and techniques to a problem defined by data that represents a challenge for its volume, speed, variety or heterogeneity, including computer, mathematical, statistical and signal processing methods.
Be able to discuss and justify in detail architectural principles and the bottlenecks of the relational managers in front of alternative storage and processing systems.
Be able to obtain the logical scheme of a data warehouse from a conceptual schema expressed in UML, detect and correct defects in it.
Be able to choose and justify the use of storage based on rows or columns.
Be able to explain and use the main mechanisms of parallel processing of queries in distributed environments, and detect bottlenecks.
Be able to justify and use distributed functional data processing environments, like MapReduce/Spark.
Data warehousing. ETL data flows. Data integration. OLAP tools. Techniques of compression and columnar storage.
Taxonomy of distributed databases. Architectures. Distributed database design (fragmentation and replication). Parallelism. Measures of scalability. Distirbuted file systems.
Distributed data processing
Importance of parallel sequential access. Synchronization barriers (Bulk Synchronous Parallel model). Distributed processing environments of functional data (MapReduce and Spark). Abstraction of distributed datasets (Resilient Distributed Datasets). Big Data architectures.
Introduction of the subject, motivation and overview of existing data management tools, their advantages and disadvantages Objectives:1 Contents:
The course consists of theory and laboratory sessions.
Theory: Reverse class techniques will be used that require the student to work on multimedia materials before class. Theory classes consist of complementary teacher explanations and problem solving.
Laboratory: Representative tools will be used for the application of theoretical concepts (for example, Indyco Builder, PotgreSQL, Pentaho Data Integration, Spark). There will also be two projects, in which students will work in teams: one on descriptive data analysis in a data warehouse and the other on predictive analysis in a Big Data environment. Consequently, there will be two deliverables outside of class hours, but students will also be assessed individually in the classroom on the knowledge gained during each of the projects.
The course has an autonomous learning component, as the students will have to work with different data management and processing tools. Apart from the support material, students should be able to resolve doubts or problems using these tools.
Be able to read and understand materials in English.
Be able to list the stages that make up the software engineering process.
Be able to understand conceptual schemas in UML.
Be able to create, query and manipulate databases with SQL.
El contingut de l'assignatura no canviarà.
La metodologia docent no canviarà. Simplement es mouran les classes de dubtes i resolucio de problemes a l'entorn virtual.
El métode d'avaluació no canviarà.
La realització d'exercicis, discussions i resolució de dubtes de les classes invertides es farà de forma remota.
Where we are
B6 Building Campus Nord
C/Jordi Girona Salgado,1-3
08034 BARCELONA Spain
Tel: (+34) 93 401 70 00