weight_initialization

Occam’s razor：簡單一刀切設置0或者1

General rule for setting weights

The general rule for setting the weights in a neural network is to set them to be close to zero without being too small.

Good practice is to start your weights in the range of $[?y,y]$ where $y=1/\sqrt{n}$
($n$ is the number of inputs to a given neuron).

Uniform Distribution：

設置分布區間為：$[?y,y]$ where $y=1/\sqrt{n}$

Normal Distribution：

設置分布區間為：a mean of 0 and a standard deviation of $y=1/\sqrt{n}$.

不設置初始化

利用網絡特殊的結構，淡化初始化的影響：

比如BN，每一層接近a mean of 0 and a standard deviation of ，自動化處理，避免了初始化的影響。

總結

以上是生活随笔為你收集整理的深度学总结：weight_initialization的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。