Spoken and Written Language Processing

• Teachers
• Weekly hours
• Competences
• Objectives
• Contents
• Activities
• Teaching methodology
• Evaluation methodology
• Bibliography
• Previous capacities

Teachers

Person in charge

• Jose Adrian Rodriguez Fonollosa ( )

Others

• Carlos Escolano Peinado ( )

Weekly hours

Competences

Technical Competences

Technical competencies

• CE5 - Design and apply techniques of signal processing, choosing between different technological tools, including those of Artificial vision, speech recognition and multimedia data processing.
• CE6 - Build or use systems of processing and comprehension of written language, integrating it into other systems driven by the data. Design systems for searching textual or hypertextual information and analysis of social networks.

Transversal Competences

Transversals

• CT5 [Avaluable] - Solvent use of information resources. Manage the acquisition, structuring, analysis and visualization of data and information in the field of specialty and critically evaluate the results of such management.
• CT6 - Autonomous Learning. Detect deficiencies in one's own knowledge and overcome them through critical reflection and the choice of the best action to extend this knowledge.
• CT7 [Avaluable] - Third language. Know a third language, preferably English, with an adequate oral and written level and in line with the needs of graduates.

Basic

• CB4 - That the students can transmit information, ideas, problems and solutions to a specialized and non-specialized public.
• CB5 - That the students have developed those learning skills necessary to undertake later studies with a high degree of autonomy

Generic Technical Competences

Generic

• CG1 - To design computer systems that integrate data of provenances and very diverse forms, create with them mathematical models, reason on these models and act accordingly, learning from experience.
• CG2 - Choose and apply the most appropriate methods and techniques to a problem defined by data that represents a challenge for its volume, speed, variety or heterogeneity, including computer, mathematical, statistical and signal processing methods.
• CG4 - Identify opportunities for innovative data-driven applications in evolving technological environments.
• CG5 - To be able to draw on fundamental knowledge and sound work methodologies acquired during the studies to adapt to the new technological scenarios of the future.

Objectives

1. Know the most important deep learning technologies of interest in the processing of oral and written language.
   Related competences: CE5, CE6, CT5, CT6, CT7, CG1, CG2, CG4, CG5, CB4, CB5,
2. The student must know the most important applications of speech and language technology.
   Related competences: CE5, CE6, CT5, CT6, CT7, CG1, CG2, CG4, CG5, CB4, CB5,
3. The student must be able to select the most appropriate speech and language technology for a particular task or application.
   Related competences: CE5, CE6, CT5, CT6, CT7, CG1, CG2, CG4, CG5, CB4, CB5,
4. Develop innovative applications that use speech technology appropriately.
   Related competences: CE5, CE6, CT5, CT6, CT7, CG1, CG2, CG4, CG5, CB4, CB5,
5. El alumno debe ser capaz de identificar los parámetros fundamentales de la voz en el dominio temporal y frecuencial
   Related competences: CE5, CT5, CT6, CT7, CG1, CB4, CB5,
6. The student must know the most important mathematical and machine learning tools for the analysis of the voice as vector quantification (VQ), Gaussian mixture models (GMM) and hidden Markov models (HMM).
   Related competences: CE5, CT5, CT6, CT7, CG1, CG2, CG4, CG5, CB4, CB5,
7. The student must know the techniques for statistical language modeling.
   Related competences: CE6, CT5, CT6, CT7, CG1, CG2, CG4, CG5, CB4, CB5,

Contents

1. Introduction to language and speech technologies and applications
   Applications of oral and written language processing. Social impact.
   Main blocks of a natural language processing system: speech recognition, natural language processing, text to speech conversion.
   Language as a sequence of words. Vector representation of words. One-hot encoding versus continuous-space representations.
   Word2vec: Continuous bag-of-words (CBOW) versus Continuous skip-gram. GloVe vectors. Structures and analogies in word vector representations.
2. Language Modeling
   Statistical modeling based on N-grams.
   Modeling with neural networks. Recurring networks Convolutional networks. Attention mechanisms: the Transformer.
3. Contextual language representations
   General purpose language representations.
   Unsupervised training. Unidirectional and bidirectional systems.
   Main architectures: ULMfit, OpenAI GPT, ELMo, BERT, XLM. Applications.
4. Neural Machine Translation
   Introduction to Machine Translation. Automatic quality evaluation: BLEU
   Neural Machine Translation.
5. Introduction to automatic speech recognition
   Pattern matching. Dynamic time warping.
   Hidden Markov models. Isolated word recognition.
   Large vocabulary continuous ASR: Acoustic modeling, Language modeling, Search.
6. Speech synthesis
   Linguistic processing.
   Prosody modeling.
   Waveform generation.
   Concatenation methods.

Activities

Activity Evaluation act

Teaching methodology

Evaluation methodology

Bibliography

Basic:

• Spoken language processing: a guide to theory, algorithm and system development - Huang, X.; Acero, A.; Hon, H.-W, Prentice Hall, 2001. ISBN: 0130226165
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991002590969706711&context=L&vid=34CSUC_UPC:VU1&lang=ca
• Deep learning - Goodfellow, I.; Bengio, Y.; Courville, A, The MIT Press, 2016. ISBN: 9780262035613
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991004107709706711&context=L&vid=34CSUC_UPC:VU1&lang=ca
• Deep learning for NLP and speech recognition - Kamath, U.; Liu, J.; Whitaker, J, Springer, 2019. ISBN: 9783030145958
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991004193579706711&context=L&vid=34CSUC_UPC:VU1&lang=ca

Complementary:

• Text-to-speech synthesis - Taylor, P, Cambridge University Press , 2009. ISBN: 9780521899277
  https://discovery.upc.edu/discovery/fulldisplay?docid=alma991003624489706711&context=L&vid=34CSUC_UPC:VU1&lang=ca

Previous capacities