Neural Networks: Learning

5?試題

1.?

You are training a three layer neural network and would like to use backpropagation to compute the gradient of the cost function. In the backpropagation algorithm, one of the steps is to update

Δ(2)ij:=Δ(2)ij+δ(3)i?(a(2))j

for every?i,j. Which of the following is a correct vectorization of this step?

Δ(2):=Δ(2)+δ(3)?(a(2))T

Δ(2):=Δ(2)+(a(2))T?δ(3)

Δ(2):=Δ(2)+δ(2)?(a(3))T

Δ(2):=Δ(2)+(a(2))T?δ(2)

2.?

Suppose?Theta1?is a 5x3 matrix, and?Theta2?is a 4x6 matrix. You set?thetaVec=[Theta1(:);Theta2(:)]. Which of the following correctly recovers?Theta2?

reshape(thetaVec(16:39),4,6)

reshape(thetaVec(15:38),4,6)

reshape(thetaVec(16:24),4,6)

reshape(thetaVec(15:39),4,6)

reshape(thetaVec(16:39),6,4)

3.?

Let?J(θ)=2θ4+2. Let?θ=1, and??=0.01. Use the formula?J(θ+?)?J(θ??)2??to numerically compute an approximation to the derivative at?θ=1. What value do you get? (When?θ=1, the true/exact derivative is?dJ(θ)dθ=8.)

8.0008

7.9992

10

8

4.?

Which of the following statements are true? Check all that apply.

Using gradient checking can help verify if one's implementation of backpropagation is bug-free.

For computational efficiency, after we have performed gradient checking to

verify that our backpropagation code is correct, we usually disable gradient checking before using backpropagation to train the network.

Computing the gradient of the cost function in a neural network has the same efficiency when we use backpropagation or when we numerically compute it using the method of gradient checking.

Gradient checking is useful if we are using one of the advanced optimization methods (such as in fminunc) as our optimization algorithm. However, it serves little purpose if we are using gradient descent.

5.?

Which of the following statements are true? Check all that apply.

If we are training a neural network using gradient descent, one reasonable "debugging" step to make sure it is working is to plot?J(Θ)?as a function of the number of iterations, and make sure it is decreasing (or at least non-increasing) after each iteration.

Suppose you have a three layer network with parameters?Θ(1)?(controlling the function mapping from the inputs to the hidden units) and?Θ(2)?(controlling the mapping from the hidden units to the outputs). If we set all the elements of?Θ(1)?to be 0, and all the elements of?Θ(2)?to be 1, then this suffices for symmetry breaking, since the neurons are no longer all computing the same function of the input.

If we initialize all the parameters of a neural network to ones instead of zeros, this will suffice for the purpose of "symmetry breaking" because the parameters are no longer symmetrically equal to zero.

Suppose you are training a neural network using gradient descent. Depending on your random initialization, your algorithm may converge to different local optima (i.e., if you run the algorithm twice with different random initializations, gradient descent may converge to two different solutions).

《新程序員》：云原生和全面數字化實踐50位技術專家共同創作，文字、視頻、音頻交互閱讀

總結

以上是生活随笔為你收集整理的Machine Learning week 5 quiz: Neural Networks: Learning的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。