Statistical Natural Language Processing

• Weekly hours
• Competences
• Objectives
• Contents
• Activities
• Teaching methodology
• Evaluation methodology
• Bibliography

Weekly hours

Competences

Generic Technical Competences

Generic

• CG1 - Capability to apply the scientific method to study and analyse of phenomena and systems in any area of Computer Science, and in the conception, design and implementation of innovative and original solutions.
• CG3 - Capacity for mathematical modeling, calculation and experimental designing in technology and companies engineering centers, particularly in research and innovation in all areas of Computer Science.

Transversal Competences

Information literacy

• CTR4 - Capability to manage the acquisition, structuring, analysis and visualization of data and information in the area of informatics engineering, and critically assess the results of this effort.

Reasoning

• CTR6 - Capacity for critical, logical and mathematical reasoning. Capability to solve problems in their area of study. Capacity for abstraction: the capability to create and use models that reflect real situations. Capability to design and implement simple experiments, and analyze and interpret their results. Capacity for analysis, synthesis and evaluation.

Basic

• CB6 - Ability to apply the acquired knowledge and capacity for solving problems in new or unknown environments within broader (or multidisciplinary) contexts related to their area of study.
• CB7 - Ability to integrate knowledges and handle the complexity of making judgments based on information which, being incomplete or limited, includes considerations on social and ethical responsibilities linked to the application of their knowledge and judgments.
• CB9 - Possession of the learning skills that enable the students to continue studying in a way that will be mainly self-directed or autonomous.

Technical Competences of each Specialization

Specific

• CEC1 - Ability to apply scientific methodologies in the study and analysis of phenomena and systems in any field of Information Technology as well as in the conception, design and implementation of innovative and original computing solutions.
• CEC2 - Capacity for mathematical modelling, calculation and experimental design in engineering technology centres and business, particularly in research and innovation in all areas of Computer Science.
• CEC3 - Ability to apply innovative solutions and make progress in the knowledge that exploit the new paradigms of Informatics, particularly in distributed environments.

Objectives

1. Justify the approppriateness of specific statistical techniques for facing specific NLP tasks.
   Related competences: CG1, CG3, CB6, CB7, CEC1, CEC2, CEC3, CTR4, CTR6,
2. Evaluating the usefulness of statistical components to be included into NLP applications for carrying out NLP tasks
   Related competences: CG3, CEC3,
3. Searching and selection of statistical NLP resources and tools able to be used in NLP tasks and applications
   Related competences: CG3, CB7, CB9, CEC1, CEC2, CEC3, CTR4,
4. Design and implementation of new NLP components, tuning of existing components, and integration into a NLP application
   Related competences: CG3, CB6, CB9, CEC2, CEC3, CTR4,

Contents

1. Introduction & basics
   NLP vs Computational Linguistics vs HLT
   Knowledge-based vs Empirical methods
   Resources
   Lexical resources
   Corpora
   Grammars
   Ontologies
2. Language Models
   Basics
   {word, class, phrase}-based models
   Information content
   entropy
   mutual information
   joint and conditional entropy
   pointwise mutual information
   Kullback-Leibler divergence (KL)
   Application to NLP tasks
   Noise channel models
   Alignment models
   Application to NLP tasks
3. Finite State Models
   Finite State Automata (FSA) and Regular grammars
   Finite State Transducers (FST)
   Finite State Probabilistic models
   Application to NLP tasks
4. Log linear & Maximum Entropy Models
   Classification problems – MLE vs MEM
   Generative and conditional (discriminative) models.
   MM and HMM.
   CRF
   Building ME models
   Maximum Entropy Markov Models (MEMM)
   Applications to NLP
5. Models for parsing
   Constituent parsing
   Stochastic Context Free Grammars (SCFG)
   Richer probabilistic models
   Applications to NLP.
   Syntactic parsing
   Semantic parsing
   Dependency parsing
6. Supervised Machine Learning for NLP
   Classification problems.
   Margin-based classifiers: Perceptron, SVM, AdaBoost.
   Kernel-based mehods.
7. Semi-supervised Learning
   Bootstrapping
8. Unsupervised Learning (Clustering)
   Similarity
   Hiereachical Clustering
   non-hierarchical clustrering
   Clustering evaluation.
9. Using statistical techniques for NLP applications
   Machine Translation (MT) in detail
   Other NLP tasks (Part of Speech (POS) tagging, Named Entity Recognition and Classification (NERC), Mention detection & tracking, Coreference resolution, Text Alignment, Lexical Acquisition, Relation Extraction, Semantic Role Labeling (SRL), Word Sense Disambiguation (WSD)) and applications (Information Extraction (IE), Information Retrieval (IR), Question Answering (Q&A), Automatic Summarization, Sentiment Analysis, and Text Classification) only sketched.

Activities

Activity Evaluation act

Teaching methodology

Evaluation methodology

Bibliography

Basic:

• Speech and Language Processing: An Introduction to Natural Language Processing, Speech Recognition and Computational Linguistics - Jurafsky, Daniel & Martin, James H., ISBN: 0131873210
  http://www.cs.colorado.edu/~martin/slp.html
• Handbook of natural language processing - Somers, Harold L; Dale, Robert, Marcel Dekker, cop.2000. ISBN: 0824790006
  http://cataleg.upc.edu/record=b1172244~S1*cat
• Foundations of statistical natural language processing - Manning, Christopher D; Schütze, Hinric, MIT Press, 1999. ISBN: 0262133601
  http://cataleg.upc.edu/record=b1165147~S1*cat
• The Oxford handbook of Computational Linguistics - Mitkov, Ruslan , 2004. ISBN: 019927634X
  http://www.chatbots.org/book/the_oxford_handbook_of_computational_linguistics/