STAT 479 PGMs | Schedule

Date	Lecture	Readings	Logistics
Module 1: Foundations of PGMs and Exact Inference
1/21	Lecture #1 (Prof. Lengerich): Course Introduction, Introduction to PGMs [ slides \| notes ]	E. Airoldi, Getting Started in Probabilistic Graphical Models
1/23	Lecture #2 (Prof. Lengerich): Statistical Review & Maximum LIkelihood Estimation [ slides \| notes ]		HW1 Out
1/28	Lecture #3 (Prof. Lengerich): A Linear view of Discriminative and Generative Models [ slides \| notes ]	MI. Jordan, Chapter 7 A. Ng, On Discriminative vs. Generative Classifiers J. Xue, Comment on On Discriminative vs. Generative Classifiers
1/30	Lecture #4 (Prof. Lengerich): Conditional Independence and Directed GMs (BNs) [ slides \| notes ]	MI. Jordan, Chapter 2.1 Koller & Friedman, Chapter 3	HW2 Out
2/4	Lecture #5 (Prof. Lengerich): Undirected GMs (MRFs) [ slides \| notes ]	MI. Jordan, Chapter 2.2 Koller & Friedman, Chapter 4
2/6	Lecture #6 (Prof. Lengerich): Exact Inference 1 - Variable Elimination [ slides \| notes ]	MI. Jordan, Chapter 2.2 Koller & Friedman, Chapter 9
2/11	No class (skipped)
2/13	Quiz
Module 2: Learning
2/18	Lecture #7 (Prof. Lengerich): Project Ideas + Learning Generalized Linear Models [ slides \| notes ]	MI. Jordan, Chapter 8, 9.1, 9.2
2/20	Lecture #8 (Prof. Lengerich): Parameter Learning in Fully-Observed BNs [ slides \| notes ]	Koller & Friedman, Chapter 17.1-17.4	HW3 Out
2/25	Lecture #9 (Prof. Lengerich): Learning Undirected GMs [ slides \| notes ]	Z. Ghahramani, Graphical models parameter learning
2/27	Lecture #10 (Prof. Lengerich): Structure Learning [ slides \| notes ]	S. Ermon, Structure learning for Bayesian networks M. Scutari, Who Learns Better Bayesian Network Structures X. Zheng, DAGs with NO TEARS	HW4 Out
3/4	Lecture #11 (Prof. Lengerich): Causal Discovery [ slides \| notes ]	K. Zhang, Learning causality and causality-related learning
3/6	Lecture #12 (Prof. Lengerich): Parameter Learning of partially observed BNs [ slides \| notes ]	MI. Jordan, Chapter 11 Koller & Friedman, Chapter 19.1-19.4	HW5 Out
3/11	Lecture #13 (Prof. Lengerich): Variational Inference [ slides \| notes ]	MI. Jordan, Chapter 15 D. Blei, Variational Inference Foundations and Innovations
3/13	Lecture #14 (Prof. Lengerich): Monte Carlo [ slides \| notes ]	MacKay, Chapter 29
3/18	Lecture #15 (Prof. Lengerich): Review [ slides \| notes ]
3/20	Exam
3/25	No class (Spring recess)
3/27	No class (Spring recess)
Module 3: Modern Probabilistic AI
4/1	Lecture #16 (Prof. Lengerich): Deep Learning from a GM Perspective [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 6.2-5, 20.3-4 Salakhutdinov and Hinton, Deep Boltzmann Machines Ranganath et al., Deep exponential families
4/3	Lecture #17 (Prof. Lengerich): CNNs, RNNs, Autoencoders [ slides \| notes ]	Pascanu, Mikolov, Bengio, On the difficulty of training recurrent neural networks
4/8	Lecture #18 : Deep Generative Models: GAN, VAEs [ slides \| notes ]	Goodfellow et al., Deep learning book, Ch. 20.9-10 Kingma and Welling, Variational Autoencoders Goodfellow et al., Generative Adversarial Nets Arora., Generative Adversarial Networks (GANs), Some Open Questions
4/10	Lecture #19 (Prof. Lengerich): Attention and Transformers [ slides \| notes ]	Vasvani et al., Attention is all you need Devlin et al., BERT - Pre-training of Deep Bidrectional Transformers for Language Understanding. Raschka, Build an LLM from Scratch 3 (video) Sanderson, Visualizing transformers and attention (video)
4/15	Lecture #20 (Prof. Lengerich): LLMs from a Probabilistic Perspective 1: Implementing a GPT from Scratch [ slides \| notes ]	Radford et al., Improving Language Understanding by Generative Pre-Training (the GPT-1 paper) Radford et al., Language Models are Unsupervised Multitask Learners (the GPT-2 paper) Brown et al., Language Models are Few-Shot Learners (the GPT-3 paper) Raschka, Build an LLM from Scratch 4 (video) Karpathy, Let's Build GPT from Scratch (video)
4/17	Lecture #21 (Prof. Lengerich): LLMs from a Probabilistic Perspective 2: Training on Unlabeled Data [ slides \| notes ]	Bi. et al, DeepSeek LLM Scaling Open-Source Language Models with Longtermism Liu et al., DeepSeekV2 A Strong, Economical, and Efficient Mixture-of-Experts Language Model Liu et al., DeepSeekV3 Technical Report
4/22	Lecture #22 (Prof. Lengerich): LLMs from a Probabilistic Perspective 3: Fine-tuning on Labeled Data [ slides \| notes ]	Raffel et al., Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Ouyang et al., Training language models to follow instructions with human feedback Li & Liang, Prefix-tuning - Optimizing continuous prompts for generation
4/24	Lecture #23 (Prof. Lengerich): New Directions in connecting LLMs to Graphical Models [ slides \| notes ]	Lengerich et al., Contextualized Machine Learning
4/29	Project Presentations
5/1	Project Presentations

1/21

Lecture #1 (Prof. Lengerich):
Course Introduction, Introduction to PGMs
[ slides | notes ]

E. Airoldi, Getting Started in Probabilistic Graphical Models

1/23

Lecture #2 (Prof. Lengerich):
Statistical Review & Maximum LIkelihood Estimation
[ slides | notes ]

HW1 Out

1/28

Lecture #3 (Prof. Lengerich):
A Linear view of Discriminative and Generative Models
[ slides | notes ]

MI. Jordan, Chapter 7
A. Ng, On Discriminative vs. Generative Classifiers
J. Xue, Comment on On Discriminative vs. Generative Classifiers

1/30

Lecture #4 (Prof. Lengerich):
Conditional Independence and Directed GMs (BNs)
[ slides | notes ]

MI. Jordan, Chapter 2.1
Koller & Friedman, Chapter 3

HW2 Out

2/4

Lecture #5 (Prof. Lengerich):
Undirected GMs (MRFs)
[ slides | notes ]

MI. Jordan, Chapter 2.2
Koller & Friedman, Chapter 4

2/6

Lecture #6 (Prof. Lengerich):
Exact Inference 1 - Variable Elimination
[ slides | notes ]

MI. Jordan, Chapter 2.2
Koller & Friedman, Chapter 9

2/11

No class (skipped)

2/13

Quiz

2/18

Lecture #7 (Prof. Lengerich):
Project Ideas + Learning Generalized Linear Models
[ slides | notes ]

MI. Jordan, Chapter 8, 9.1, 9.2

2/20

Lecture #8 (Prof. Lengerich):
Parameter Learning in Fully-Observed BNs
[ slides | notes ]

Koller & Friedman, Chapter 17.1-17.4

HW3 Out

2/25

Lecture #9 (Prof. Lengerich):
Learning Undirected GMs
[ slides | notes ]

Z. Ghahramani, Graphical models parameter learning

2/27

Lecture #10 (Prof. Lengerich):
Structure Learning
[ slides | notes ]

S. Ermon, Structure learning for Bayesian networks
M. Scutari, Who Learns Better Bayesian Network Structures
X. Zheng, DAGs with NO TEARS

HW4 Out

3/4

Lecture #11 (Prof. Lengerich):
Causal Discovery
[ slides | notes ]

K. Zhang, Learning causality and causality-related learning

3/6

Lecture #12 (Prof. Lengerich):
Parameter Learning of partially observed BNs
[ slides | notes ]

MI. Jordan, Chapter 11
Koller & Friedman, Chapter 19.1-19.4

HW5 Out

3/11

Lecture #13 (Prof. Lengerich):
Variational Inference
[ slides | notes ]

MI. Jordan, Chapter 15
D. Blei, Variational Inference Foundations and Innovations

3/13

Lecture #14 (Prof. Lengerich):
Monte Carlo
[ slides | notes ]

MacKay, Chapter 29

3/18

Lecture #15 (Prof. Lengerich):
Review
[ slides | notes ]

3/20

Exam

3/25

No class (Spring recess)

3/27

No class (Spring recess)

4/1

Lecture #16 (Prof. Lengerich):
Deep Learning from a GM Perspective
[ slides | notes ]

Goodfellow et al., Deep learning book, Ch. 6.2-5, 20.3-4
Salakhutdinov and Hinton, Deep Boltzmann Machines
Ranganath et al., Deep exponential families

4/3

Lecture #17 (Prof. Lengerich):
CNNs, RNNs, Autoencoders
[ slides | notes ]

Pascanu, Mikolov, Bengio, On the difficulty of training recurrent neural networks

4/8

Lecture #18 :
Deep Generative Models: GAN, VAEs
[ slides | notes ]

Goodfellow et al., Deep learning book, Ch. 20.9-10
Kingma and Welling, Variational Autoencoders
Goodfellow et al., Generative Adversarial Nets
Arora., Generative Adversarial Networks (GANs), Some Open Questions

4/10

Lecture #19 (Prof. Lengerich):
Attention and Transformers
[ slides | notes ]

Vasvani et al., Attention is all you need
Devlin et al., BERT - Pre-training of Deep Bidrectional Transformers for Language Understanding.
Raschka, Build an LLM from Scratch 3 (video)
Sanderson, Visualizing transformers and attention (video)

4/15

Lecture #20 (Prof. Lengerich):
LLMs from a Probabilistic Perspective 1: Implementing a GPT from Scratch
[ slides | notes ]

Radford et al., Improving Language Understanding by Generative Pre-Training (the GPT-1 paper)
Radford et al., Language Models are Unsupervised Multitask Learners (the GPT-2 paper)
Brown et al., Language Models are Few-Shot Learners (the GPT-3 paper)
Raschka, Build an LLM from Scratch 4 (video)
Karpathy, Let's Build GPT from Scratch (video)

4/17

Lecture #21 (Prof. Lengerich):
LLMs from a Probabilistic Perspective 2: Training on Unlabeled Data
[ slides | notes ]

Bi. et al, DeepSeek LLM Scaling Open-Source Language Models with Longtermism
Liu et al., DeepSeekV2 A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Liu et al., DeepSeekV3 Technical Report

4/22

Lecture #22 (Prof. Lengerich):
LLMs from a Probabilistic Perspective 3: Fine-tuning on Labeled Data
[ slides | notes ]

Raffel et al., Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Ouyang et al., Training language models to follow instructions with human feedback
Li & Liang, Prefix-tuning - Optimizing continuous prompts for generation

4/24

Lecture #23 (Prof. Lengerich):
New Directions in connecting LLMs to Graphical Models
[ slides | notes ]

Lengerich et al., Contextualized Machine Learning

4/29

Project Presentations

5/1

Project Presentations