This curriculum provides an overview of free online resources for learning about deep learning. It includes courses, books, and even important people to follow.
If you only want to do one thing, do this:
Train an MNIST network with PyTorch.
CS231n: Convolutional Neural Networks for Visual Recognition
CS231n Class Notes:
Introduction to Gradient Based Learning
Fast.ai’s Online Courses:
Full Stack Deep Learning (Especially their infrastructure section.):
Deep Learning Book - the definitive deep learning textbook (Goodfellow, Bengio, and Courville):
Recurrent Neural Networks - a very easy to understand research paper by Alex Graves:
Convolutional Neural Networks:
Deep Learning for NLP:
CS224d: Deep Learning for Natural Language Processing: http://cs224d.stanford.edu
Often, the easiest way to build a mental model of a field is to read the work of the important contributors in chronological order. This is a helpful list of important players in the field of deep learning.
Geoff Hinton (University of Toronto)
Andrew Ng (Stanford, Coursera, Baidu, Landing AI)
Worked with Jeff Dean + Hinton @ Google's "Google Brain Team"
Yann LeCun (Courant Institute NYU, now Meta) - Specializes in Convnets
Yoshua Bengio (University de Montreal) - I love his work and papers, very easy to read.
Keywords: manifold learning and representation learning.
Ian Goodfellow - invented GANs (yea)
Andrej Karpathy (Tesla’s AI head) - great work with CNNs
Alex Graves - great work with LSTMs, Neural Turing Machines, Differentiable Neural Computer
Oriol Vinyols - co-author of seq2seq learning and knowledge distillation
Early Deep Learning Papers I learned from:
To Recognize Shapes, First Learn to Generate Images
Practical recommendations for gradient-based training of deep architectures
Bengio's Representation Learning
Ng's Large Scale Learning Paper
Thumbnail is Figure 4 from the original AlexNet paper: ImageNet Classification with Deep Convolutional Neural Networks.