Backprop

moving towards the direction of gradient --> Optimize the loss

setting up the parameters --> forward pass --> compute loss --> change the parameters slightly in the direction of gradient of parameters --> backward pass

Gradient of neuron with respect to loss function:

1. how much loss changes when change in neuron weight, gradient reflects the growth sign (negative or positive) and value of growth is called slope of a function.