CS3264 · Foundations of Machine Learning

Announcements

The latest first. Anything time-critical also goes out on Piazza.

12 Oct 2026 Updated schedule for Weeks 10–12. We will cover autoencoders and latent variable models on 20 October, variational inference and VAEs on 27 October, and diffusion models on 3 November. Kernel methods, SVMs and EM are omitted this semester. The tutorial topics are updated in the schedule below.

12 Oct 2026 Assignment 2 is out: Repair the Model. 15% of the final grade, individual work, due Friday 13 November 2026, 23:59 — the same day as Assignment 1. Take a question that Qwen3-1.7B gets wrong, design a reusable repair, and test whether it holds up on new variants and on questions the model already got right. The handout has everything, including a notebook with recorded examples.

9 Sep 2026 Assignment 1 is out: Stump the Model. 15% of the final grade, individual work, due Friday 13 November 2026, 23:59. Write six single-number questions on course topics that a fixed small language model gets wrong, and explain why. The handout has everything, including a Colab notebook that runs the exact grading model.

Before the semester Welcome to CS3264. Sign up for Piazza — that is where announcements, questions and resources live. Tutorials start in Week 3.

What this course is

Machine learning is the study of models and algorithms for finding patterns in data. CS3264 is about the machinery underneath — the part that transfers when the tools change.

By the end you should be able to

Describe what ML is, especially supervised and unsupervised learning.
Explain fundamental techniques, how they work — and when they will not.
Design your own ML methods.

This course is not

a pure mathematics or statistics course
a programming course
a complete survey of machine learning

You will go beyond calling libraries: reading state-of-the-art methods, adapting them, and debugging models that misbehave.

Prerequisites

CS2109S, one of CS2040/CS2040S/CS2040C/CS2040DE/CS2040HS, one of MA1101R/MA1311/MA1508E/MA1513/MA1522/MA2001, and one of EE2012/EE2012A/MA2116/MA2116T/MA2216/ST2131/ST2334 — each at grade D or better. Precluded by CS3244 and CG3201.

In practice: comfort with linear algebra, vector calculus and probability. If the first two lectures feel fast, work through Chapters 2–5 of Mathematics for Machine Learning.

Course notes: Mathematical prerequisites — notation, algebra, linear algebra, derivatives, probability and optimisation, and where each appears in CS3264.

Schedule

Tentative — this will change. Topics and pacing adapt as we go.

Wk	Tue	Lecture	Tutorial (Wed)	Reading	Materials
1	11 Aug	Introduction + Regression I	No tutorial; work through the maths prerequisites prerequisites notes	PRML Ch. 1	slides · revised slides · video · notes
2	18 Aug	Regression II — Probabilistic Modelling	No tutorial; work through the maths prerequisites prerequisites notes	PRML §1.2, §3.1.1, §3.3	slides · revised slides · video · notes
3	25 Aug	Classification I — Linear Classification	Probability & linear algebra; pseudoinverse tutorial 1 sheet · solutions	PRML §4.1.3, §4.3.2, §4.3.4	slides · revised slides · video · notes
4	1 Sep	Classification II — Naive Bayes	Log-odds, multi-class tutorial 2 sheet · solutions	PRML §4.2	slides · revised slides · video · notes
5	8 Sep	Overfitting & Bias–Variance	Norms and regularisers; naive Bayes ↔ logistic regression tutorial 3 sheet · solutions	PRML §3.2	slides · revised slides · video · notes
6	15 Sep	Computational Learning Theory (PAC)	Bias–variance; midterm scope tutorial 4 sheet · solutions	Learning From Data Ch. 1–2	slides · revised slides · video · notes
—	19–27 Sep	Recess week
7	29 Sep	MIDTERM — in the lecture slot, LT15	Midterm debrief	—
8	6 Oct	Neural Networks I	PAC problems tutorial 5 sheet · solutions	PRML §5.1–5.3	slides · video · notes
9	13 Oct	Neural Networks II — CNNs, RNNs, Transformers, SGD	Backpropagation and stochastic gradient descent tutorial 6 sheet · solutions	PRML §5.5.6	slides · video · notes
10	20 Oct	Unsupervised I — Clustering and Representation Learning	Multivariate Gaussians tutorial 7 sheet · solutions	PRML §9.1, §12.1.1–12.1.2, §12.4.2; Deep Learning Ch. 14 introduction, §14.1–14.2 Optional: ISL Ch. 12 For the tutorial: PRML §2.3 through §2.3.3	slides · video · notes
11	27 Oct	Unsupervised II — Variational Inference and VAEs	Latent variable models tutorial 8 sheet · solutions	PRML §1.6.1; §10.1 opening (before §10.1.1); Auto-Encoding Variational Bayes Understanding Diffusion Models: A Unified Perspective (Luo, tutorial) §2.1–2.2 Tutorial on Variational Autoencoders (Doersch)	slides · video · notes
12	3 Nov	Unsupervised III — Diffusion Models	KL divergence and variational inference tutorial 9 sheet · notebook · open in Colab	Denoising Diffusion Probabilistic Models Understanding Diffusion Models: A Unified Perspective (Luo, tutorial)	slides · video · notes
13	10 Nov	Exam-prep recitation (TAs) Instructor away (CoRL)	KL/VAE problems; final exam recap	—	semester review
—	14–20 Nov	Reading week
—	25 Nov	FINAL EXAM · 5–7pm · venue TBA Provisional

Readings are worth doing before the lecture. Slides and recordings are posted here as they are ready; recordings are also on Panopto via Canvas. Tutorial sheets go up on Tuesdays for the following week’s Wednesday session — work through them beforehand.

Assignments

Two assignments, 15% each. Both are individual work, and both are due on Friday 13 November 2026, 23:59 through Canvas.

Assignment 1 — Stump the Model

Write six pen-and-paper questions on course topics that a fixed small language model (Qwen3-1.7B) gets wrong, with your own solutions and a short prediction of why it fails.

Handout · test notebook · open in Colab

Assignment 2 — Repair the Model

Take a question that Qwen3-1.7B gets wrong, build a reusable repair, and test it against a baseline and a control, on new variants, and on questions the model already gets right.

Handout · notebook · JSON template · open in Colab

Assessment

Both exams are closed book. No cheat sheet for the midterm; a two-page cheat sheet for the final.

30%

Assignments

10%

Tutorials

20%

Midterm

40%

Final exam

A1 and A2 due Fri 13 Nov Weeks 3–13 Tue 29 Sep, 4–6pm Wed 25 Nov, 5–7pm · provisional

Assignments — 30%

Two individual assignments, 15% each, both due Friday 13 Nov 2026, 23:59. Details and files are in Assignments.

Midterm — 20%

Tue 29 Sep 2026, 4–6pm · LT15. In class, on paper, MCQ and fill-in-the-blank. Starts at 4pm sharp. No make-up paper.

No cheat sheet and no notes. Exact scope announced on Piazza nearer the date.

Final exam — 40%

Wed 25 Nov 2026, 5–7pm. Two hours, paper exam, cumulative. You may bring a two-page cheat sheet you prepare yourself. Venue TBA.

Provisional — taken from the published AY26/27 timetable. The official NUS exam timetable is the authority; confirm before booking anything. Note it falls early in the exam period, about two weeks after the last lecture.

Key dates

11 Aug 2026 — First lecture
26 Aug 2026 — First tutorial (Week 3)
24 Aug – 27 Sep 2026 — dropping gives a “W” grade
28 Sep 2026 — dropping gives an “F” grade

29 Sep 2026 — Midterm
10 Nov 2026 — Last lecture
13 Nov 2026 — Assignments 1 and 2 due
14–20 Nov 2026 — Reading week
25 Nov 2026 — Final exam Provisional

Resources

Everything here is free.

Main textbook

Pattern Recognition and Machine Learning — Christopher Bishop. The weekly readings in the schedule refer to this one.

Do not overlook the appendices — they are a compact reference for most of the mathematics the chapters assume: B probability distributions, C matrix identities, D calculus of variations, E Lagrange multipliers.

Alternate textbook — and the basics

Mathematics for Machine Learning — Deisenroth, Faisal & Ong. Useful two ways.

As a second treatment when Bishop’s presentation of a topic does not land — the same material, explained differently, is often all it takes.

And for the background itself: Chapters 2–5 cover essentially all the linear algebra, vector calculus and probability this course assumes.

Past years’ Collaborative Notes

Earlier cohorts built one shared set of notes covering the whole syllabus — probability and vector calculus through to PAC bounds, kernels, EM and VAEs.

AY2025/26 notes — PDF, 37 pages. Written by students, so treat them as a study aid rather than an authority: verify anything you intend to rely on.

They may not be brought into either exam this year. The midterm is fully closed book; the final allows a two-page cheat sheet you write yourself.

Code

Lecture notebooks are on Google Colab; links appear with each lecture. They are written to be read, not just run — the implementations follow the equations as we derive them in the course, rather than the fastest library call.

Policies

Academic integrity

The principle is simple: your graded work should be your own. NUS has a strict policy on plagiarism; cases are reported to the department, and if found guilty you will fail the course.

Fine: discussing ideas and approaches; using public information in your own words, with attribution.
Not fine: copying verbatim; sharing code or write-ups.

There are always better options — ask on Piazza, arrange a consultation, or hand in something incomplete. All of those are recoverable.

On using LLMs

You may use them. We would rather set a sensible policy than pretend otherwise.

Use them as a tutor — explanations, analogies, hints when stuck. Try opening with: “You are a teaching assistant in a machine learning class. Your goal is to help me understand. Do not give me the answer directly.”

Do not paste their output into graded work. That is plagiarism.

They are good at this material now — which is the problem. A derivation you did not do teaches you nothing, and both exams are on paper. The errors that remain are subtle: a dropped factor of ½, a convention that is not ours, a bound stated almost right — the kind you can only catch if you could have done it yourself.

Tutorials

Five one-hour slots on Wednesdays in BIZ2-0302, from Week 3 — T06 (9am), T01 (10am), T02 (11am), T04 (1pm), T05 (2pm), taught by Shashank and Oscar. Attendance is taken and counts for 10%; you may miss two tutorials without penalty.

T03 (12pm) is closed. Other slots may still be merged after add/drop depending on final numbers; if your slot moves you will be reassigned.

Announcements

What this course is

By the end you should be able to

This course is not

Prerequisites

Schedule

Assignments

Assignment 1 — Stump the Model

Assignment 2 — Repair the Model

Assessment

Assignments — 30%

Midterm — 20%

Final exam — 40%

Key dates

Resources

Main textbook

Alternate textbook — and the basics

Past years’ Collaborative Notes

Code

Policies

Academic integrity

On using LLMs

Tutorials

Teaching staff