ECE 414/517: Reinforcement Learning (Fall 2026)

Time and Location

Lectures: Tuesday and Thursday, 12:55–2:10 PM
Location: MKB-405
Office hours: Thursdays, 2:15–3:00 PM (Min Kao Building, Room 612) or by appointment

Instructors

Instructor: Fei Liu
Teaching assistant: Farong Wang

Course Description

This course develops the foundations of reinforcement learning: how an agent can learn to make sequential decisions through interaction with an environment. We will build the subject from multi-armed bandits and Markov decision processes through dynamic programming, Monte Carlo methods, temporal-difference learning, tabular control, planning, function approximation, and policy gradients. The final portion of the course introduces advanced reinforcement learning, reward design, partially observable settings etc.

The emphasis is on understanding the assumptions, derivations, and behavior of core algorithms—not simply applying software libraries. Examples and assignments will connect the theory to complete learning systems and evaluation.

Canvas: UTK Canvas
Syllabus: PDF

Learning Objectives

By the end of the course, students should be able to:

• formulate sequential decision problems as bandits or MDPs;
• derive and apply Bellman equations and dynamic programming methods;
• compare Monte Carlo and temporal-difference prediction and control;
• explain on-policy, off-policy, model-free, and model-based learning;
• analyze approximation, policy-gradient, and deep-RL methods; and
• design reproducible experiments and diagnose learning behavior.

Prerequisites

Familiarity with probability, linear algebra, basic optimization, and programming is expected. There is no separate mathematics or coding foundations unit; students should refresh these topics independently as needed.

Grading

The tentative grading breakdown is:

Late Policy: Up to 24-hours extensions, with a 25% penalty

Primary References

• Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, second edition.
• David Silver, Reinforcement Learning Course, University College London.