Neural Networks and Deep Learning

by Michael Nielson

Read here Read at LibreTextsPDF

Neural networks are one of the most beautiful programming paradigms ever invented. In the conventional approach to programming, we tell the computer what to do, breaking big problems up into many small, precisely defined tasks that the computer can easily perform. By contrast, in a neural network we don't tell the computer how to solve our problem. Instead, it learns from observational data, figuring out its own solution to the problem at hand.

Description from LibreTexts.

Publisher
Y Combinator Research
Subjects
Engineering, Computer Science
Licence
CC BY-NC 3.0, as LibreTexts states it. Non-commercial use only.
Who can read
Tweens profiles and older, judged from its title and subjects

Contents

LibreTexts writes equations in TeX, and here they show as TeX; the PDF has them typeset.

  1. Front Matter
  2. TitlePage
  3. InfoPage
  4. Table of Contents
  5. Licensing
  6. Acknowledgements
  7. 1: Using neural nets to recognize handwritten digits
  8. 1.1: Introduction
  9. 1.2: Perceptrons
  10. 1.3: Sigmoid neurons
  11. 1.4: The architecture of neural networks
  12. 1.5: A simple network to classify handwritten digits
  13. 1.6: Learning with gradient descent
  14. 1.7: Implementing our network to classify digits
  15. 1.8: Toward deep learning
  16. 2: How the Backpropagation Algorithm Works
  17. 2.1: Introduction
  18. 2.2: Warm up- a fast matrix-based approach to computing the output from a neural network
  19. 2.3: The backpropagation algorithm
  20. 3: Improving the way neural networks learn
  21. 3.1: The cross-entropy cost function
  22. 3.2: Overfitting and regularization
  23. 3.3: Weight initialization
  24. 3.4: Handwriting recognition revisited- the code
  25. 3.5: How to choose a neural network's hyper-parameters?
  26. 3.6: Other techniques
  27. 4: A visual proof that neural nets can compute any function
  28. 4.1: Two caveats
  29. 4.2: Universality with one input and one output
  30. 4.3: Many input variables
  31. 4.4: Extension beyond sigmoid neurons
  32. 4.5: Fixing up the step functions
  33. 4.6: Section 6-
  34. 5: Why are deep neural networks hard to train?
  35. 5.1: The vanishing gradient problem
  36. 5.2: Unstable gradients in more complex networks
  37. 6: Deep Learning
  38. 6.1: Introducing convolutional networks
  39. 6.2: Convolutional neural networks in practice
  40. 6.3: The code for our convolutional networks
  41. 6.4: Recent progress in image recognition
  42. 6.5: Other approaches to deep neural nets
  43. 6.6: On the future of neural networks
  44. 7: Appendix- Is there a simple algorithm for intelligence?
  45. 7.1: Is there a simple algorithm for intelligence?
  46. Back Matter
  47. Index
  48. Glossary
  49. Detailed Licensing