Loss Functions in Deep Learning

A Guide on the Concept of Loss Functions in Deep Learning — What they are, Why we need them…

Source: https://unsplash.com/photos/yNvVnPcurD8

This in-depth article addresses the questions of why we need loss functions in deep learning and which loss functions should be used for which tasks.

In Short: Loss functions in deep learning are used to measure how well a neural network model performs a certain task.

Table of Content

1. Why do we need Loss Functions in Deep Learning?
2. Mean Squared Error Loss Function
3. Cross-Entropy Loss Function
4. Mean Absolute Percentage Error
5. Take-Home-Message

1. Why do we need Loss Functions in Deep Learning?

Before we discuss different kinds of loss functions used in Deep Learning, it would be a good idea to address the question of why we need loss functions in the first place.

I think you must be familiar by now with the mathematical operations which are happening inside a neural network. Basically, there are just two:

• Forward Propagation
• Backpropagation with Gradient Descent

While forward propagation refers to the computational process of predicting an output for a given input vector x, backpropagation and gradient descent describe the process of improving the weights and biases of the network in order to make better predictions.

Recap: Forward Propagation

For a more in-depth explanation of Forward Propagation and Backpropagation in neural networks, please refer to my other article What is Deep Learning and How does it work?

For a given input vector x the neural network predicts an output, which is generally called a prediction vector y.

Fig, 1. Feedforward neural network. Source: Author’s own image.

The equations which describe the mathematics that is happening during the computation of the prediction vector loos as follows: