Syllabus

DateTopicRecommended ReadingsDeadlines
Week 1 — Mon Aug 25Introduction
Week 1 — Wed Aug 27Lecture - Transformers
Week 2 — Mon Sep 1Labor Day - No Class
Week 2 — Wed Sep 3Lecture - GPT3++Sign up sheet released
Week 3 — Mon Sep 8Lecture - GPT3++Submit Sign up for presentations, 11.59 pm
Week 3 — Wed Sep 10Lecture - Post training 2
Week 4 — Mon Sep 15Lecture - Post training 2
Week 4 — Wed Sep 17Student Presentations: Evaluation News Summarization and Evaluation in the Era of GPT-3
Week 5 — Mon Sep 22Student Presentations: Evaluation Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Week 5 — Wed Sep 24Student Presentations: Evaluation This blog + “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models
Week 6 — Mon Sep 29Student Presentations: Long Context Method: Longformer: The Long-Document TransformerProject Proposal Due: Sep 29, 11.59pm
Week 6 — Wed Oct 1Student Presentations: Long Context Evaluation: RULER: What's the Real Context Size of Your Long-Context Language Models?
Week 7 — Mon Oct 6Guest Lecture
Week 7 — Wed Oct 8Tanya Traveling - No Class
Week 8 — Mon Oct 13Indigenous Peoples' Day - No Class
Week 8 — Wed Oct 15Student Presentations: Long Context Method Method: Efficient streaming language models with attention sinks
Week 9 — Mon Oct 20Student Presentations: Long Context Method Method: SnapKV: LLM Knows What You are Looking for Before GenerationArea review Due: Oct 22, 11.59pm
Week 9 — Wed Oct 22Student Presentations: Data: Data Engineering for Scaling Language Models to 128K Context
Week 10 — Mon Oct 27Student Presentations: Long Context Evaluation: One Thousand and One Pairs: A "novel" challenge for long-context language models
Week 10 — Wed Oct 29Student Presentations: Reasoning Method: STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning
Week 11 — Mon Nov 3Student Presentations: Reasoning Method: Let's Verify Step by Step
Week 11 — Wed Nov 5Student Presentations: Reasoning Method: Iterative Reasoning Preference OptimizationProject Check-in Due Nov 5, 11.59pm
Week 12 — Mon Nov 10Student Presentations: Reasoning DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Week 12 — Wed Nov 12Student Presentations: Reasoning Analysis: Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRsArea review Due: Nov 14, 11.59pm
Week 13 — Mon Nov 17Student Presentations: Factuality Evaluation: FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Week 13 — Wed Nov 19Student Presentations: Factuality Analysis: Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Week 14 — Mon Nov 24Student Presentations: Factuality Analysis: HALoGEN: Fantastic LLM Hallucinations and Where to Find ThemArea review Due: Nov 25, 11.59pm
Week 14 — Wed Nov 26Thanksgiving Break — No Class
Week 15 — Mon Dec 1Course Retrospective - Tanya
Week 15 — Wed Dec 3Project Presentations
Week 16 — Mon Dec 8Project Presentations