2: How the Backpropagation Algorithm Works — Read

From LibreTexts, under CC BY-NC 3.0, as the source states it; its equations are written in TeX, which the PDF shows typeset. This page at LibreTexts.

In the last chapter we saw how neural networks can learn their weights and biases using the gradient descent algorithm. There was, however, a gap in our explanation: we didn't discuss how to compute the gradient of the cost function. That's quite a gap! In this chapter I'll explain a fast algorithm for computing such gradients, an algorithm known as backpropagation.

2.1: Introduction
The backpropagation algorithm was originally introduced in the 1970s, but its importance wasn't fully appreciated until a famous 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams. That paper describes several neural networks where backpropagation works far faster than earlier approaches to learning, making it possible to use neural nets to solve problems which had previously been insoluble. Today, the backpropagation algorithm is the workhorse of learning in neural networks.
2.2: Warm up- a fast matrix-based approach to computing the output from a neural network
2.3: The backpropagation algorithm

This page titled 2: How the Backpropagation Algorithm Works was last modified on Tue, 01 Dec 2020 03:47:43 GMT and is shared under a CC BY-NC 3.0 license and was authored, remixed, and/or curated by Michael Nielson via source content that was edited to the style and standards of the LibreTexts platform.

Contents

Text