Long Nguyen

This is a gentle introduction to Neural Networks. The goal of this Video Lecture Series is to write a Python program from scratch that recognizes handwritten digits.

The course attempts to make the material as accessible as possible. The programming prerequisites are minimal: 1) Basic Python or some programming experience in another language. 2) Numpy. Both of these are covered in the Jupyter Notebook tutorials below. The video lectures cover basic Python and Numpy necessary for writing the program.

The mathematical requirements are also minimal. A precalculus background is all that is required. The course, however, will cover some basic multivariable calculus, including the discussion of partial derivatives, the chain rule and the gradient, necessary to understand Gradient Descent and backpropagation.

This course is currently being taught on Udemy(free) and on Youtube.

Prerequisites

The essentials of Python and Numpy needed for the course are covered in the following three Jupyter Notebooks tutorials.

1. Python Crash Course Part I (html, ipynb): Numbers, Booleans, Strings, Conditionals and Loops
2. Python Crash Course Part 2 (html, ipynb): Functions Containers: Lists, Tuples.
3. Introduction to Numpy (html, ipynb): Numpy: Working with Arrays, Matrix operations, Broadcasting.

Lecture 1(Video): Introduction to Neural Networks

Overview. MNIST Dataset. Training set. Affine and sigmoid functions.

• Python Code From Lecture Video: Jupyter Notebook, HTML
• Homework: lec1_hw_and_solutions.zip

Lecture 2 Functions and Their Computational Graphs

Real and vector-valued functions. Matrix Operations. Computational Graphs. N-layer Neural Networks.

Lecture 3: Formalizing the Problem

Computing the loss and objective functions. Vectorization. Broadcasting. Training the classifier.

Lecture 4 What is a derivative? A gradient?

Intuitive definitions of the derivative. Partial Derivatives. The gradient.

Lecture 5 Gradient Descent and Backpropagation

The Gradient Descent Algorithm. Four Equations of Backpropagation. Mini-batch Gradient Descent.

Lecture 6 Writing an Image Recognition in Python.