Author: Bonnin, Rodolfo
-
loc 1786 - Neural network layers The single layer perceptron can be generalized to many layers connected to eachother, but there is an issue remaining;the representing function is a linear combination of the inputs, and the perceptron being just a kind of linear classifier,there is no possibility of correctly fitting a nonlinear function.
-
loc 2158 - In case of overfitting, the model is specifically adjusted to the training dataset, so it will not be useful for generalization.
-
loc 2162 - The main advantage of this method is that it avoids all neurons in a layer to synchronously optimize their weights. This adaptation made in random groups avoids all the neurons converging to the same goals, thus decorrelating the adapted weights.
-
loc 2164 - second property discovered in the dropout application is that the activation of the hidden units becomes sparse, which is also a desirable characteristic.