Mathematics concept required for Deep Learning

Why is Math required for Deep Learning?
Interested people who have the thirst to learn more about the concept behind a deep learning algorithm need to tackle Mathematics in some path of the way or another. Math is the core concept from which Deep Learning algorithms are built upon and is used to express the idea that seems quite obvious, but these are unexpectedly hard to elaborate and once it is elaborated properly, we can gain a proper understanding of the problem that we are given to solve. In this article, we are going to discuss in detail about the math required for Deep Learning. Now if there is a spark a light inside you, to learn more about deep learning then start with these Math topics:

1. Geometry and Linear Algebra
   • Geometry of Vectors
   • Angles and Dot Products with Cosine similarity
   • Hyperplanes
   • The geometry of Linear Transformation
   • Rank of Matrix
   • Linear Dependence
   • Invertibility
   • Determinant

   Learn Linear Algebra here and basics on Geometry here

2. Matrix Decomposition
   • Finding Eigenvalues and Eigenvectors
   • Decomposing Matrices
   • Operations on Eigendecomposition
   • Single Value Decomposition
   • Principal Component Analysis
   • Matrix Approximation
   • Eigendecomposition and Diagonalization of Symmetric Matrices
   • Gershgorin circle theorem
   • LU Decomposition, QR Decomposition/Factorization
   • Symmetric Matrices, Orthogonalization and Orthonormalization

   Learn about Matrix Decomposition here

3. Single Variable Calculus
   • Differential Calculus
   • Rules of calculus

   Learn calculus here

4. Probability and Distributions
   • Sum Rule, Product Rule, and Bayes Theorem
   • Gaussian Distribution
   • Discrete and Continuous Probabilities
   • Combinatorics
   • Conditional and Joint Distributions
   • Bernoulli Distribution
   • Discrete Uniform Distribution
   • Binomial Distribution
   • Poisson Distribution
   • Continuous Uniform Distribution
   • Gaussian Distribution
   • Exponential Family Distribution

   Refer this for videos on Probability and for different distributions here.

5. Naive Bayes
   • Optical Characters Recognition
   • Probabilistic Model for Classification
   • Naive Bayes Classifier

   To learn more about this refer to GeeksForGeek’s article here.

6. Multivariate Calculus
   • Higher-dimensional Differentiation
   • Multivariate chain rule
   • Backpropagation Algorithm
   • Gradient, Gradient Descent, and its geometry
   • Integral Calculus, Partial Derivatives
   • Vector-Values Functions
   • Jacobian, Laplacian, Lagrangian Distribution

   Refer this book to learn more about it

7. Integral Calculus
   • Theorem of Calculus
   • Sign Conventions
   • Geometric interpretation
   • Multiple integrals Concepts and Change of Variables

   For basics of integral calculus refer here.

8. Random Variables
   • The concept of Discrete to Continuous in Random Variables
   • Means, Variances, Standard Deviation and its concept on a continuum
   • Probability Density Function
   • Cumulative Distribution Function
   • Joint Density Function
   • Marginal Distribution
   • Covariance and Correlation

   An in-depth guide to this topic here

9. Statistics
   • Evaluation and Comparison of Estimators
   • Conducting Hypothesis Tests
   • Constructing Confidence intervals

   Refer here to learn more about it.

10. Probability on Maximum Likelihood
    • Principle of Maximum Likelihood with examples
    • Numerical optimization and Negative Log-Likelihood
    • Maximum Likelihood for Continuous Variables
    • Moment Generating Function
    • Prior and Posterior, Maximum a Posteriori Estimation, Sampling Methods

To learn more about it refer here

• Information Theory (Optional For beginners) Learn about it here

After getting good at the above mathematical topics you can go ahead and get your hands dirty with these topics.

1. Some advanced Mathematical Topics
   • Hamiltonian Calculus
   • Halleys Calculus
   • Complex Numbers
   • Quaternions
   • Sedenions
   • Quadratic Function
   • Np Problems
   • Advanced Probabilities and Statistics

   A one-stop guide for all the above topics here

2. Three reasons why mathematics will help in your future with a career in Deep Learning field:-
   • Math help in selecting a correct algorithm considering its complexity, training time, feature and accuracy
   • Approximate the right confidence interval and unpredictability.
   • Help in selecting an algorithm’s acceptance plan and in choosing its parameter setting.
3. Some of the interesting application requiring Deep Learning Algorithm: –
   • Suppose you have your great-great-grandfather black and while a picture may be captured in the early 20th century, well now with the help of Deep Neural Network its now possible to colourize that black and white picture and for a surprise, this is also possible with a video.

Stanford documentation about the above here

• Pixel recursive super-resolution being developed by Google Brain Researchers has come up with this deep learning neural network, and this is capable of predicting a somewhat precise image of an almost blurred image.

Google publication on this here.

• Lip reading developed by Oxford University is a deep learning neural network capable of reading the lips of a person and convert that directly into the text and doesn’t even need the sound of a person speaking.

Stanford documentation on this here.

• Deep Learning neural network is now capable of detecting the location of the picture where it was clicked on and display it on a map.

Refer to PlaNet documentation here.

• Some endangered whale species are being detected with convolution neural network and this is where deep learning concepts are being implemented to save them.

Refer here to learn more.

• Self-driving cars, which can detect traffic and choose an optimal path is already developed.

Refer to Stanford documentation here.

• Apart from this Deep learning algorithms are being implemented everywhere, for example, earthquake prediction, music composition, entertainment, healthcare, and of course Robotics.

More here

To learn more about Deep Learning and neural network refer to this link below.

References: –

• Introduction to Artificial Neutral Networks
• Introduction Deep Learning
• More amazing application on deep learning
• MIT notes for Mathematics
• A book for an in-depth guide to math in deep learning
• Difference between Machine Learning vs Artificial intelligence vs Deep Learning