CMPUT 651: Topics in Artificial Intelligence

Deep Learning for NLP

Fall 2019

Instructor: Lili Mou

Course Description

This course introduces deep learning (DL) techniques for natural language processing (NLP).
Contrary to other DL4NLP courses, we would have a whirlwind tour of all neural
architectures (e.g., CNNs, RNNs, attention) in a few lectures. Then, we would make
significant efforts in learning structured prediction using Bayesian and Markov networks,
with applications of sequential labeling, syntactic parsing, and sentence generation. In this
process, we will also see how such traditional methods can be combined with and improve
a plain neural network.


No DL or NLP background is required. They will be self-contained.

Tentative Syllabus

  1. Neural network basics
  2. Structured Prediction
  3. Sentence Generation
  4. Discrete Latent Space
The syllabus is subject to changes.


01. NLP Tasks and Linear Classification [slides]
02. Deep Neural Network [slides]
03. Word Embeddings and Language Modeling [slides]
04. CNNs, RNNs, etc. [slides]
05. Seq2Seq Models and Attention Mechanism [slides]
06. hmm [slides]
07. em...hmm [slides]
08. MRF & CRF [slides]