Course Schedule
The schedule is tentative and may be adjusted as the semester progresses. Course materials will be posted through this site and Canvas.
No class
Tutorial / Q&A
Project milestone
| Week | Date | Lecture | Assignment |
|---|---|---|---|
| 1 | Tue, Aug 18 | Course introductionAgent-environment interaction, rewards, examples, and challenges | — |
| 1 | Thu, Aug 20 | Multi-armed banditsAction-value methods, exploration versus exploitation, and nonstationary problems | — |
| 2 | Tue, Aug 25 | Markov Decision Processes IStates, actions, transition models, Markov property, and policies | — |
| 2 | Thu, Aug 27 | Dynamic Programming I: Value Functions and Bellman EquationsState- and action-value functions, Bellman expectation equations, and optimal value functions | — |
| 3 | Tue, Sep 1 | Dynamic Programming II: Policy Evaluation and Policy IterationIterative policy evaluation, policy improvement, and policy iteration | — |
| 3 | Thu, Sep 3 | Dynamic Programming III: Value IterationBellman optimality equations, value iteration, generalized policy iteration, and examples | — |
| 4 | Tue, Sep 8 | Monte Carlo PredictionModel-free value estimation, first-visit and every-visit methods, and sample-average returns | — |
| 4 | Thu, Sep 10 | In-class tutorial: Quizzes 1–3Guided quiz practice and review | — |
| 5 | Tue, Sep 15 | Monte Carlo ControlExploring starts, on-policy control, and soft policies | Homework 1 + Coding Assignment 1 released Mon, Sep 14 |
| 5 | Thu, Sep 17 | Temporal-difference Prediction and n-step TDTD(0), bootstrapping, MC–TD comparison, and n-step prediction | — |
| 6 | Tue, Sep 22 | Temporal-difference Control and n-step TDSARSA, expected SARSA, Q-learning, n-step SARSA, and off-policy n-step learning | — |
| 6 | Thu, Sep 24 | TA-led coding tutorial: tabular TD controlImplement SARSA and Q-learning, run controlled experiments, and compare their learning behavior | — |
| 7 | Tue, Sep 29 | No class — IROS travelOptional guided review | — |
| 7 | Thu, Oct 1 | No class — IROS travel | — |
| 8 | Tue, Oct 6 | No class — Fall Break | — |
| 8 | Thu, Oct 8 | Homework 1 and coding assignment Q&A / review sessionTroubleshoot Homework 1 and Coding Assignment 1, and review common issues | Homework 1 + Coding Assignment 1 DUEBring your questions |
| 9 | Tue, Oct 13 | Combined project (building a complete RL system) — midterm milestoneProject integration | Bring a laptop |
| 9 | Thu, Oct 15 | Combined project (building a complete RL system) — midterm milestone System demonstrations, testing, feedback, and milestone review Final Project Coding Tutorial | Homework 2 + Coding Assignment 2 releasedFinal project phase begins: train an RL agent in simulationBring a laptop |
| 10 | Tue, Oct 20 | Value-function Approximation I: PredictionGradient descent review, state aggregation, linear features, objectives, gradient Monte Carlo, and semi-gradient TD | — |
| 10 | Thu, Oct 22 | Value-function Approximation II: Control and StabilityApproximate action values, semi-gradient SARSA, off-policy learning, instability, and the deadly triad | — |
| 11 | Tue, Oct 27 | Policy Gradients IParameterized stochastic policies, policy objectives, score functions, and REINFORCE | — |
| 11 | Thu, Oct 29 | Policy Gradients IIBaselines, variance reduction, actor–critic, and continuing tasks | — |
| 12 | Tue, Nov 3 | No class — Election Day | — |
| 12 | Wed, Nov 4 | Assignment deadline | Homework 2 + Coding Assignment 2 DUE |
| 12 | Thu, Nov 5 | Policy Gradient Algorithms coding labImplement policy gradient algorithms and compare their learning behavior | Homework 3 + Coding Assignment 3 released |
| 13 | Tue, Nov 10 | Planning and Learning I: Dyna and Background PlanningModels, direct and indirect learning, Dyna-Q, simulated experience, and background planning | Final project progress check (submit a short summary report) |
| 13 | Thu, Nov 12 | Planning and Learning II: Decision-time PlanningBackground versus decision-time planning, heuristic search, rollout algorithms, and Monte Carlo Tree Search | — |
| 14 | Tue, Nov 17 | Deep reinforcement learningDQN, experience replay, target networks, representation learning, and training stability | — |
| 14 | Thu, Nov 19 | Reward design and learning from preferencesReward hypothesis, reward shaping, inverse reinforcement learning, and preference-based feedback | — |
| 15 | Tue, Nov 24 | RL beyond the Markov propertyPartial observability, state representation, memory, belief states, and recurrent agents | — |
| 15 | Thu, Nov 26 | No class — Thanksgiving Recess | — |
| 15 | Fri, Nov 27 | Assignment deadline | Homework 3 + Coding Assignment 3 DUE |
| 16 | Tue, Dec 1 | Final project poster and demo sessionsPresent the completed system, trained RL agent, experimental results, and lessons learned | Project poster and live demo |
UTK calendar notes: Full-session classes end Tuesday, December 1; Study Day is Wednesday, December 2; final exams run December 3–9. Final project deadlines will be confirmed in the class.