Open Data

Credits
6
Types
Specialization compulsory (Data Science)
Requirements
This subject has not requirements

Department
ESSI
Big Data is traditionally defined with the three V's: Volume, Velocity and Variety. Traditionally, Big Data has been associated with Volume (e.g., the Hadoop ecosystem) and recently Velocity has earned its momentum (especially, with the arrival of Stream processors such as Spark). However, even if Variety has been part of the Big Data definition, how to tackle Variety in real-world projects is yet not clear and there are no standarized solutions (such as Hadoop for Volume or Spark for Velocity) for this challenge.

In this course the student will be introduced to advanced database technologies, modeling techniques and methods for tackling Variety for decision making. We will also explore the difficulties that arise when combining Variety with Volume and / or Velocity. The focus of this course is on the need to enrich the available data (typically owned by the organization) with external repositories (special attention will be paid to Open Data), in order to gain further insights into the organization business domain. There is a vast amount of examples of external data to be considered as relevant in the decision making processes of any company. For example, data coming from social networks such as Facebook or Twitter; data released by governmental bodies (such as town councils or governments); data coming from sensor networks (such as those in the city services within the Smart Cities paradigm); etc.

This is a new hot topic without a clear and established (mature enough) methodology. For this reason, it requires rigorous thinking, innovation and a strong technical background in order to master the inclusion of external data in an organization decision making processes. Accordingly, this course focuses on three main aspects:

1.- Technical aspect. This represents the core discussion in the course and includes:
- dealing with semi-structured or non-structured data (as in the Web),
- the effective use of metadata to understand external data as by means of Linked Data,
- mastering the main formalisms (mostly coming from the Semantic Web) to enrich the data with metadata (ontology languages, RDF, XML, etc.),
- determine relevant sources, apply and use semantic mechanisms to automate the addition (potentially integration), linkage and / or cross of data between heterogeneous data sources,
- refining and visualizing Open Data

2.- Ethic and social aspects, which includes:
- data ownership aspects,
- ethics and,
- identifying knowing legal frameworks (such as that of the LOPDP in Spain)

3.- Entrepreneurship and innovation, which includes:
- working on the visionary aspect to boost new analytical perspectives on a business domain by considering external sources and,
- developing added value to current systems by means of (such) external data

Teachers

Person in charge

  • Oscar Romero Moral ( )

Weekly hours

Theory
2
Problems
0
Laboratory
1
Guided learning
0
Autonomous learning
5.33

Competences

Generic Technical Competences

Generic

  • CG4 - Capacity for general and technical management of research, development and innovation projects, in companies and technology centers in the field of Informatics Engineering.

Transversal Competences

Entrepreneurship and innovation

  • CTR1 - Capacity for knowing and understanding a business organization and the science that rules its activity, capability to understand the labour rules and the relationships between planning, industrial and commercial strategies, quality and profit. Capacity for developping creativity, entrepreneurship and innovation trend.

Teamwork

  • CTR3 - Capacity of being able to work as a team member, either as a regular member or performing directive activities, in order to help the development of projects in a pragmatic manner and with sense of responsibility; capability to take into account the available resources.

Technical Competences of each Specialization

Specific

  • CEC1 - Ability to apply scientific methodologies in the study and analysis of phenomena and systems in any field of Information Technology as well as in the conception, design and implementation of innovative and original computing solutions.
  • CEC3 - Ability to apply innovative solutions and make progress in the knowledge that exploit the new paradigms of Informatics, particularly in distributed environments.

Objectives

  1. Determine relevant external sources to be considered in the decision making processes in order to generate added value in the day-by-day processes
    Related competences: CG4, CEC3, CTR1, CTR3,
  2. Master the main semantic-aware formalisms to enable semantic modeling
    Related competences: CEC1,
  3. Integrate, combine and refine semi-structured or non-structured data mostly coming from the Web into decisional systems
    Related competences: CEC1, CEC3,
  4. Reinforce team work capabilities in order to develop innovative solutions by means of complementing the organization data with external data
    Related competences: CG4, CTR1, CTR3,
  5. Enable effective open / linked data visualization
    Related competences: CEC3,
  6. Recognise the social and legal aspects of open data
    Related competences: CG4, CTR1,

Contents

  1. Introduction
    Big Data & Business Intelligence 2.0. The relevance of external data. Open Data.
  2. Linked Data and Semantic Modeling
    Definition. The four rules. The 5 stars of linked data. The relevance of metadata.
  3. XML and XML databases
    XML. XPATH. XQUERY. Foundations on XML Databases
  4. JSON and Document-Stores
    Principles on document-stores. JSON and BSON
  5. RDF, Graph Databases and Triplestores
    RDF. RDFS. SPARQL. Foundations on graph-databases. Foundations on triplestores
  6. Ontology Languages and Ontology-Based Data Access
    OWL. Datalog. Description Logics. Foundations on Ontology-based Data Access
  7. Refining, Combining and Integrating External Data
    XSLT. Restructuring data. Data integration. Stream Processing. Mashups. Data Warehousing 2.0. Distributed Systems
  8. Visual Analytics
    Theory of visualization. Visual representations. User Experience
  9. Legal Aspects and Innovation and Enterpreunership
    Ethics. Legal frameworks. LOPDP. Adding value to day-by-day processes

Activities

Lectures

During lectures the main concepts will be discussed. Lectures will combine master lectures and active / cooperative learning activities. The student is meant to have a pro-active attitude during active / cooperative learning activities. During master lectures, the student is meant to listen, take notes and ask questions.
Theory
28
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
28
Objectives: 3 2 1 5 4 6
Contents:

Hands-on Session

The student will be asked to practice the different concepts introduced in the lectures. This includes problem solving either on the computer or on paper.
Theory
0
Problems
0
Laboratory
14
Guided learning
0
Autonomous learning
28
Objectives: 3 2 5 4
Contents:

Seminars

The students will be asked to choose a case study on which integrating external data would add value to the current organization processes. They will need to determine what data should be considered and what is the benefit of this new solution. The students are asked to sketch a solution and prepare a demo, which will be presented to their classmates.
Theory
6
Problems
0
Laboratory
4
Guided learning
0
Autonomous learning
32
Objectives: 3 2 1 5 4 6
Contents:

Final Exam

Written exam of the theoretical concepts introduced along the course.
Theory
2
Problems
0
Laboratory
0
Guided learning
0
Autonomous learning
8
Objectives: 3 2 1 5
Contents:

Teaching methodology

The course comprises theory, lab sessions and seminars.

Theory: These lectures comprise the teacher's explanations and constitute the main part of the course. The students will also have some contents to be read and prepared outside the classroom and will be asked to participate in cooperative learning activities.

Laboratory: Mainly, the lab sessions will be dedicated to the practice (with and without computer) of the concepts introduced in the theory lectures, by means of markable exercises that will be done during the class time. Some tools will be used for the design and practice on a specific DBMS.

Seminar: The students will have to prepare a practical seminar by themselves.This seminar is focused on developing innovative and entrepreneurial aspects related to the inclusion of external data into the organization decisional systems.

Evaluation methodology

Final mark = 30% P + 30% EX + 30% S + 10% C

EX = Final exam mark
P = Course practice
S = Seminars and hand-outs
C = Peer evaluation

S: This mark corresponds to the seminars students must prepare and present in front of their classmates. It also includes the hand-outs to be delivered during the course. Based on the seminar presentation and the delivered materials, the lecturer will assign a mark.

P: Each group will prepare a course practice during the whole term. There will be several hand-outs during the course that the lecturer will pick up and assess. These hand-outs account for 50% of P. The rest of the mark (50%) will rely on the final presentation in the last week of the course.

C: During the course practice each student will interact with two other students. Since the practice is meant to entail several working hours each student will peer-mark his / her teammates twice (at midterm and at the end of the term). The lecturer will assign a mark to each student according to the peer-marking received by his / her teammates.

Bibliografy

Basic:

Previous capacities

The student must be familiar with basics on databases and data modeling. Programming skills are also mandatory.