深度學習（2）回歸問題

• 一. 問題提出與解析
• • 1. Machine Learning
  • 2. Continuous Prediction
  • 3. Linear Equation
  • 4. With Noise?
  • 5. Find $w^{\prime }$，$b^{\prime }$
  • 6. Gradient Descent
• 二. 回歸問題實戰
• • 1. 步驟
  • 2. Step1: Compute Loss
  • 3. Step2: Compute Gradient and update
  • 4. Step3: Set $w=w^{\prime }$and loop
  • 5. 代碼

一. 問題提出與解析

1. Machine Learning

• make decisions
• going left/right $\to$ discrete
• increase/decrease $\to$ continuous

2. Continuous Prediction

• $f_{\theta }:x\to y$
• $x:inputdata$
• $f(x):prediction$
• $y:realdata,ground?truth$

3. Linear Equation

• y=w*x+b
• 1.567=w*1+b
• 3.043=w*2+b

$\to$ Closed Form Solution

• w=1.477
• b=0.089

4. With Noise?

• y=w*x+b+?
• ? ~ N(0,1)
• 1.567=w*1+b+eps
• 3.043=w*2+b+eps
• 4.519=w*2+b+eps
• …
  $\to$
• Y=(WX+b)

For Example

• w?
• b?

5. Find $w^{\prime }$，$b^{\prime }$

• $[(WX+b?Y){]}^2$
• $loss={\sum }_i(w?x_i+b?y_i{)}^2$
• $Minimizeloss$
• $w^{\prime }?x+b^{\prime }\to y$

6. Gradient Descent

(1) 1-D
$$w^{\prime }=w^{\prime }?lr?\frac{dy}{dw}$$

$$x^{\prime }=x?0.005?\frac{dy}{dw}$$
可以看到，函數的導數始終指向函數值變大的方向，因此，如果要求$loss$函數的極小值的話，就需要沿導數的反方向前進，即$?lr?\frac{dy}{dw}$，衰減因子$lr$的引入是為了防止步長變大，跨度太大。
(2) 2-D

Find$w^{\prime },b^{\prime }$

• $loss={\sum }_i(w?x_i+b?y_i{)}^2$
• 分別對w和b求偏導數，然后沿著偏導數的反向前進，即:
  • $w^{\prime }=w?lr?\frac{?loss}{?w}$
  • $b^{\prime }=b?lr?\frac{?loss}{?b}$
• $w^{\prime }?x+b^{\prime }\to y$

Learning Process

Loss surface

二. 回歸問題實戰

1. 步驟

(1) 根據隨機初始化的$w,x,b,y$的數值來計算$LossFunction$;
(2) 根據當前的$w,x,b,y$的值來計算梯度;
(3) 更新梯度，將$w^{\prime }$賦值給$w$，如此往復循環;
(4) 最后面的$w^{\prime }$和$b^{\prime }$就會作為模型的參數。

2. Step1: Compute Loss

共有100個點，每個點有兩個維度，所以數據集維度為$[100,2]$，按照$[(x_0,y_0),(x_1,y_1),\dots ,(x_{99},y_{99})]$排列，則損失函數為:
$$loss=[(w_0{x}_0+b_0?y_0){]}^2+[(w_0{x}_1+b_0?y_1){]}^2+?+[(w_0{x}_{99}+b_0?y_{99}){]}^2$$
即:
$$loss=\sum _i(w?x_i+b?y_i{)}^2$$
初始值設$w_0=b_0=0$。

(1) b和w的初始值都為0，points是傳入的100個點，是data.csv里的數據;
(2) len(points)就是傳入數據點的個數，即100; range(0, len(points))就代表從0循環到100;
(3) x=points[i, 0]表示取第i個點中的第0個值，即第一個元素，相當于p[i][0]; 同理，y=points[i, 1]表示取第i個點中的第1個值，即第二個元素，相當于p[i][1];
(4) totalError為總損失值，除以是len(points)是平均損失值。

3. Step2: Compute Gradient and update

$$los{s}_0=(w{x}_0+b?y_0{)}^2$$
$$\frac{?los{s}_0}{?w}=2(w{x}_0+b?y_0)x_0$$
$$\frac{?loss}{?w}=2\sum (w{x}_i+b?y_i)x_i$$
$$\frac{?loss}{?b}=2\sum (w{x}_i+b?y_i)$$
$$w^{\prime }=w?lr?\frac{?loss}{?w}$$
$$b^{\prime }=b?lr?\frac{?loss}{?b}$$

4. Step3: Set $w=w^{\prime }$and loop

$$w\leftarrow w^{\prime }$$
$$b\leftarrow b^{\prime }$$

計算出最終的w和b的值就可以帶入模型進行預測了:
$$w^{\prime }x+b^{\prime }\to predict$$

5. 代碼

import numpy as np# y = wx + b def compute_error_for_line_given_points(b, w, points):totalError = 0for i in range(0, len(points)):x = points[i, 0]y = points[i, 1]# computer mean-squared-errortotalError += (y - (w * x + b)) ** 2# average loss for each pointreturn totalError / float(len(points))def step_gradient(b_current, w_current, points, learningRate):b_gradient = 0w_gradient = 0N = float(len(points))for i in range(0, len(points)):x = points[i, 0]y = points[i, 1]# grad_b = 2(wx+b-y)b_gradient += (2 / N) * ((w_current * x + b_current) - y)# grad_w = 2(wx+b-y)*xw_gradient += (2 / N) * x * ((w_current * x + b_current) - y)# update w'new_b = b_current - (learningRate * b_gradient)new_w = w_current - (learningRate * w_gradient)return [new_b, new_w]def gradient_descent_runner(points, starting_b, starting_w, learning_rate, num_iterations):b = starting_bw = starting_w# update for several timesfor i in range(num_iterations):b, w = step_gradient(b, w, np.array(points), learning_rate)return [b, w]def run():points = np.genfromtxt("data.csv", delimiter=",")learning_rate = 0.0001initial_b = 0 # initial y-intercept guessinitial_w = 0 # initial slope guessnum_iterations = 1000print("Starting gradient descent at b = {0}, w = {1}, error = {2}".format(initial_b, initial_w,compute_error_for_line_given_points(initial_b, initial_w, points)))print("Running...")[b, w] = gradient_descent_runner(points, initial_b, initial_w, learning_rate, num_iterations)print("After {0} iterations b = {1}, w = {2}, error = {3}".format(num_iterations, b, w,compute_error_for_line_given_points(b, w, points)))if __name__ == '__main__':run()

運行結果如下:

可以看到，在$w=0,b=0$的時候，損失值$error\approx 5565.11$;
在1000輪迭代后，$w\approx 1.48,b\approx 0.09$，損失值$error\approx 112.61$，要大大小于原來的損失值。

參考文獻:
[1] 龍良曲:《深度學習與TensorFlow2入門實戰》

總結

以上是生活随笔為你收集整理的深度学习（2）回归问题的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。