Deep Neural Networks

The era between the AlexNet moment (2012) and "Attention Is All You Need" (2017) — and the foundational machinery of every modern model.

Reading path

1. Building Blocks — perceptron → MLP → backprop → losses → initialization.
2. Optimization — SGD, Adam, learning-rate schedules.
3. Regularization & Generalization — dropout, batch norm, augmentation, double descent.
4. Convolutional Networks — convolution, LeNet/AlexNet, VGG/Inception/ResNet.
5. Recurrent & Sequence Models — RNN, LSTM/GRU, seq2seq, Bahdanau attention.
6. Generative Models (pre-diffusion) — autoencoders, VAEs, GANs, normalizing flows, PixelRNN/CNN.
7. Graph Neural Networks — GCN, message passing, GAT.
8. Reinforcement Learning — full track imported from NoteNextra · CSE510.

Continue from here

• For vision-specific deep architectures, jump to Computer Vision →.
• For language models and the post-attention era, jump to Large Language Models → and then The Transformer Era →.