Machine Learning Fundamentals
Modern AI rests on a handful of ideas that get named constantly and explained rarely. What they actually mean: how neural networks learn, what makes transformers work, and the optimization behind both — drawn from MIT, Stanford, Berkeley, and Princeton course materials.
Introduction to Machine Learning
Settles the first question in any ML project: is this classification or regression, do you have labels or not, and how will you know whether the result is any good. Covers supervised and unsupervised learning and the evaluation metrics everything else gets judged by.
University Course Materials
MIT
6.390 Introduction to Machine Learning · Tommi Jaakkola, Regina Barzilay
- Chapter 1: Introduction →
What a learning algorithm is, and the vocabulary the rest of the notes assume
- Chapter 2: Regression →
Predicting a number: least squares and the first look at a loss function
- Chapter 4: Classification →
Predicting a label: linear classifiers and logistic regression
Stanford
CS229 Machine Learning · Andrew Ng
- Lecture notes, Part I: Supervised Learning →
Chapters 1-6 of the course notes: linear regression, logistic regression, generative models, kernels, SVMs
- Supervised learning notes (archive) →
The standalone supervised-learning chapter, if you want a shorter PDF than the full notes
UC Berkeley
CS189/289A Introduction to Machine Learning · Jonathan Shewchuk
- Lecture 1: Introduction, Classification, Train/Validate/Test →
How a problem gets framed and how you split data to measure a result honestly
- Full-semester lecture notes →
All 25 lectures as one PDF
Princeton
COS 324 Introduction to Machine Learning · Sanjeev Arora, Danqi Chen
- Chapter 1: Linear Regression →
The simplest supervised model, worked end to end
- Chapter 2: Statistical Learning - What It Means to Learn →
Train versus test error, and why fitting the data is not the goal
- Chapter 4: Linear Classification →
Moving from predicting numbers to predicting labels
Cornell
CS4780 Machine Learning for Intelligent Systems · Kilian Weinberger
- Lecture 1: ML Setup →
Feature vectors, labels, loss, and the train/test split
- Lecture 2: k-Nearest Neighbors →
The first classifier, and what it shows about distance and dimensionality
Caltech
CS156 Learning From Data · Yaser Abu-Mostafa
- Lecture 1: The Learning Problem (video) →
What it means for a machine to learn from data
- Lecture 1 slides →
Slides for the same lecture
Neural Networks
How a network turns being wrong into a corrected weight. Backpropagation and gradient descent are the mechanism, and activation functions like ReLU are what let a network model anything more interesting than a straight line.
University Course Materials
MIT
6.390 Introduction to Machine Learning
- Chapter 6: Neural Networks →
From a single unit to a multi-layer network, with backpropagation derived step by step
- Appendix B: Optimizing Neural Networks →
Momentum, Adadelta, Adam, and the details of batch normalization
Princeton
COS 324 Introduction to Machine Learning
- Chapter 10: Introduction to Deep Learning →
Why stacked nonlinear layers can represent things a linear model cannot
- Chapter 11: Feedforward Neural Network and Backpropagation →
The forward pass, the backward pass, and the chain rule that connects them
UC Berkeley
CS189/289A Introduction to Machine Learning
- Lecture 16: Neural Networks and Backpropagation →
How the gradient of the loss gets pushed back through every layer
- Lecture 17: Vanishing Gradients, ReLUs, Softmax →
Why activation choice decides whether a deep network trains at all
Cornell
CS4780 Machine Learning for Intelligent Systems
- Lecture 20: Neural Networks →
Network structure and the training procedure, as lecture slides
Caltech
CS156 Learning From Data
- Lecture 10: Neural Networks (video) →
The perceptron, multilayer networks, and the backpropagation algorithm
- Lecture 10 slides →
Slides for the same lecture
Deep Learning
Why stacking layers needs its own techniques. What convolutional networks exploit about images, what the hidden layers end up representing, and what batch normalization and residual connections are actually fixing once a network gets deep.
University Course Materials
MIT
6.390 Introduction to Machine Learning
- Chapter 7: Convolutional Neural Networks →
Filters, pooling, and the weight sharing that makes image models tractable
- Chapter 8: Representation Learning →
What the hidden layers end up encoding, and how to reuse it
Stanford
CS229 Machine Learning
- Deep learning notes →
Neural network notation, vectorization, and backpropagation in one PDF
Princeton
COS 324 Introduction to Machine Learning
- Chapter 12: Convolutional Neural Network →
The convolution operation and the architectures built from it
UC Berkeley
CS189/289A Introduction to Machine Learning
- Lecture 19: Convolutional Neural Networks →
What a convolutional layer assumes about images
- Lecture 23: Batch Normalization, ResNets, AdamW →
The techniques that make very deep networks trainable
Transformers & Attention
What is happening inside every language model you use. Self-attention decides which tokens matter to each other, multi-head attention runs that comparison several ways at once, and positional encoding restores the word order attention alone discards.
University Course Materials
MIT
6.390 Introduction to Machine Learning
- Chapter 9: Transformers →
Query, key, and value worked through from scratch, then assembled into a transformer block
Stanford
CS229 Machine Learning
- Chapter 17: Large Language Models →
Sections 17.3 and 17.4 of the course notes cover the transformer architecture and attention variants
Princeton
COS 324 Introduction to Machine Learning
- Chapter 17: Deep Learning for Natural Language Processing →
Word embeddings through to attention-based language models
UC Berkeley
CS189/289A Introduction to Machine Learning
- Lecture 20: Transformers →
Fall 2025 lecture slides on self-attention and transformer architectures
External Resources
Optimization & Training
Why training is finicky and what the knobs actually do. How stochastic gradient descent differs from the plain version, what Adam adapts, and why the learning rate is the single setting most likely to decide whether a model converges at all.
University Course Materials
MIT
6.390 Introduction to Machine Learning
- Chapter 3: Gradient Descent →
Plain gradient descent, stochastic gradient descent, and what the step size controls
- Appendix B: Optimizing Neural Networks →
How momentum, Adadelta, and Adam adapt the step size as training goes
Princeton
COS 324 Introduction to Machine Learning
- Chapter 3: Optimization via Gradient Descent →
The update rule, convergence, and how the learning rate decides the outcome
UC Berkeley
CS189/289A Introduction to Machine Learning
- Lecture 3: Gradient Descent →
Gradient descent as the workhorse behind most of the models in the course
- Lecture 18: Neural Network Training Heuristics →
Initialization, learning rate schedules, and the settings that decide whether training works
Cornell
CS4780 Machine Learning for Intelligent Systems
- Lecture 7: Gradient Descent and Optimization →
Gradient descent, Newton's method, and adaptive step sizes
Caltech
CS156 Learning From Data
- Lecture 12: Regularization (video) →
Why constraining a model during training improves what it does afterward
- Lecture 11: Overfitting (video) →
The failure mode regularization exists to prevent
Practical ML
The gap between a model that scores well and a system that works. Choosing an evaluation split that does not flatter you, tuning hyperparameters without overfitting to your own test set, handling imbalanced data, and what changes once it has to run in production.
University Course Materials
Stanford
CS229 Machine Learning
- Advice for Applying Machine Learning →
Andrew Ng on diagnosing what is actually wrong before changing the model
- Chapters 8-9: Generalization, Regularization and Model Selection →
Bias-variance, cross-validation, and how to pick a model without fooling yourself
Cornell
CS4780 Machine Learning for Intelligent Systems
- Lecture 11: Model Selection →
Cross-validation and choosing hyperparameters without leaking the test set
- Lecture 12: Bias and Variance →
Deciding whether more data or a bigger model is the fix
UC Berkeley
CS189/289A Introduction to Machine Learning
- Lecture 12: Bias-Variance Decomposition →
Where prediction error actually comes from
- Lecture 13: Ridge Regression and Lasso →
The two regularizers you will reach for most often