Author: Phil Kim
-
loc 520 - Keep in mind that the hidden layer becomes ineffective when the hidden nodes have linear activation functions.
-
loc 660 - The mini batch method , when it selects an appropriate number of data points, obtains the benefits from both methods: speed from the SGD and stability from the batch.
-
loc 662 - Now, let’s delve a bit into the SGD , batch, and mini batch in terms of the epoch. The epoch is briefly introduced in the “Training of a Single-Layer Neural Network: Delta Rule” section. As a recap, the epoch is the number of completed training cycles for all of the training data. In the batch method, the number of training cycles of the neural network equals an epoch, as shown in Figure 2-18. This makes perfect sense because the batch method utilizes all of the data for one training process.