CMPUT 651: Topics in Artificial Intelligence

Deep Learning for NLP

Fall 2019

Instructor: Lili Mou

Course Description

This course introduces deep learning (DL) techniques for natural language processing (NLP).
Contrary to other DL4NLP courses, we would have a whirlwind tour of all neural
architectures (e.g., CNNs, RNNs, attention) in a few lectures. Then, we would make
significant efforts in learning structured prediction using Bayesian and Markov networks,
with applications of sequential labeling, syntactic parsing, and sentence generation. In this
process, we will also see how such traditional methods can be combined with and improve
a plain neural network.


No DL or NLP background is required. They will be self-contained.


  1. Neural network basics
  2. Structured Prediction
  3. Advanced Topics


01. NLP Tasks and Linear Classification [slides]
02. Deep Neural Network [slides]
03. Word Embeddings and Language Modeling [slides]
04. CNNs, RNNs, etc. [slides]
05. Seq2Seq Models and Attention Mechanism [slides]
06. hmm [slides]
07. em...hmm [slides]
08. MRF & CRF [slides]
09. Discrete Latent Variables [slides]
10. Sentence Generation [slides]

Note: Lectures for Part II were derived on the whiteboard.