日韩性视频-久久久蜜桃-www中文字幕-在线中文字幕av-亚洲欧美一区二区三区四区-撸久久-香蕉视频一区-久久无码精品丰满人妻-国产高潮av-激情福利社-日韩av网址大全-国产精品久久999-日本五十路在线-性欧美在线-久久99精品波多结衣一区-男女午夜免费视频-黑人极品ⅴideos精品欧美棵-人人妻人人澡人人爽精品欧美一区-日韩一区在线看-欧美a级在线免费观看

歡迎訪問(wèn) 生活随笔！

生活随笔

當(dāng)前位置： 首頁(yè) > 编程资源 > 编程问答 >内容正文

编程问答

第1章 数学基础和机器学习问题（范数+矩阵迹+矩阵求导+机器学习框架）

發(fā)布時(shí)間：2023/12/16 编程问答 21 豆豆
生活随笔 收集整理的這篇文章主要介紹了 第1章 数学基础和机器学习问题（范数+矩阵迹+矩阵求导+机器学习框架） 小編覺(jué)得挺不錯(cuò)的,現(xiàn)在分享給大家,幫大家做個(gè)參考.

俯視機(jī)器學(xué)習(xí)

能看透人生，卻看不透數(shù)學(xué)，有什么好處！

第1章 數(shù)學(xué)基礎(chǔ)和機(jī)器學(xué)習(xí)問(wèn)題

1. 向量及矩陣

本課程討論的向量默認(rèn)都是列向量。向量和矩陣都只討論實(shí)數(shù)情況。

1.1 向量?jī)?nèi)積

對(duì)向量 x,y∈Rnx,y\in R^nx,y∈Rn，其內(nèi)積
<x,y>=xTy=∑i=1nxiyi<x,y>=x^Ty = \sum_{i=1}^n x_i y_i <x,y>=xTy=i=1∑n?xi?yi?

內(nèi)積滿足交換律，即 xTy=yTxx^T y = y^T xxTy=yTx .

1.2 矩陣的跡

對(duì)方陣 A∈Rn×nA\in R^{n\times n}A∈Rn×n，其跡為對(duì)角線元素之和。
tr(A)=∑i=1nAiitr(A) = \sum_{i=1}^n A_{ii} tr(A)=i=1∑n?Aii?

  • 性質(zhì)（假設(shè)矩陣滿足相關(guān)乘法或轉(zhuǎn)置規(guī)則）：

tr(AT)=tr(A)tr(A+B)=tr(A)+tr(B)tr(AB)=tr(BA)tr(ABC)=tr(BCA)=tr(CAB)tr(A^T) = tr(A) \\ tr(A+B) = tr(A) + tr(B) \\ tr(AB) = tr(BA) \\ tr(ABC) = tr(BCA) = tr(CAB) tr(AT)=tr(A)tr(A+B)=tr(A)+tr(B)tr(AB)=tr(BA)tr(ABC)=tr(BCA)=tr(CAB)

  • 方陣的跡等于其特征值之和。

1.3 矩陣內(nèi)積

兩個(gè)尺寸相關(guān)的矩陣的內(nèi)積，定義為兩矩陣逐元素相乘后的和，即
<A,B>=∑i,jAijBij=tr(ATB)<A,B> = \sum_{i,j}A_{ij} B_{ij} = tr(A^T B) <A,B>=i,j∑?Aij?Bij?=tr(ATB)

2. 范數(shù)

2.1 定義

如果函數(shù) f:Rn→Rf:R^n \rightarrow Rf:Rn→R 的定義域?yàn)?domf=Rn{\rm dom f}=R^ndomf=Rn 而且滿足以下條件，則稱 fff 是范數(shù)：
- 非負(fù)性： 對(duì)任意的 x∈Rnx\in R^nx∈Rn，都有 f(x)≤0f(x)\le 0f(x)≤0，且 f(x)=0f(x)=0f(x)=0 時(shí)，必有 x=0x=0x=0；
- 齊次性： 對(duì)任意的 x∈Rn,t∈Rx\in R^n, t\in Rx∈Rn,t∈R，都有 f(tx)=∣t∣f(x)f(tx)=|t|f(x)f(tx)=∣t∣f(x)；
- 滿足三角不等式：對(duì)于任意的 x,y∈Rnx,y\in R^nx,y∈Rn，均有 f(x+y)≤f(x)+f(y)f(x+y)\le f(x) + f(y)f(x+y)≤f(x)+f(y)

用記號(hào) f(x)=∥x∥f(x) = \Vert x\Vertf(x)=∥x∥ 表示范數(shù)。

理解：

  • 范數(shù)本質(zhì)上是一個(gè)函數(shù)。
  • 度量向量（或矩陣）的“長(zhǎng)度”。
  • 主要探討向量范數(shù)。

2.2 幾種常見(jiàn)的向量范數(shù)

  • ppp范數(shù)

∥x∥p=(∣x1∣p+?+∣xn∣p)1/p\Vert x\Vert_p = (|x_1|^p + \cdots + |x_n|^p)^{1/p} ∥x∥p?=(∣x1?∣p+?+∣xn?∣p)1/p

  • 1范數(shù)

∥x∥1=(∣x1∣+?+∣xn∣)\Vert x\Vert_1 = (|x_1| + \cdots + |x_n|) ∥x∥1?=(∣x1?∣+?+∣xn?∣)

  • 2范數(shù)

∥x∥2=(∣x1∣2+?+∣xn∣2)1/2\Vert x\Vert_2 = (|x_1|^2 + \cdots + |x_n|^2)^{1/2} ∥x∥2?=(∣x1?∣2+?+∣xn?∣2)1/2

  • 無(wú)窮范數(shù)

∥x∥∞=max?{∣x1∣,?,∣xn∣}\Vert x\Vert_\infty = \max\{|x_1|, \cdots, |x_n|\} ∥x∥∞?=max{∣x1?∣,?,∣xn?∣}

  • 0范數(shù)（不是范數(shù)）：向量中非零元素個(gè)數(shù)。

2.3 向量范數(shù)可視化

2.4 矩陣的Frobenius范數(shù)

矩陣 A∈Rm×nA\in R^{m\times n}A∈Rm×n 的 Frobenius 范數(shù)定義為
∥A∥F=(tr(ATA))1/2=(∑i=1m∑j=1nAij2)1/2\Vert A\Vert_F = (tr(A^T A))^{1/2} = \left(\sum_{i=1}^m \sum_{j=1}^n A_{ij}^2\right)^{1/2} ∥A∥F?=(tr(ATA))1/2=(i=1∑m?j=1∑n?Aij2?)1/2

3. 向量和矩陣的導(dǎo)數(shù)

3.1 求導(dǎo)規(guī)則

  • 向量和標(biāo)量導(dǎo)數(shù)：
    (?a?x)i=?a?xi\left( \frac{\partial a}{\partial \boldsymbol x} \right)_i = \frac{\partial a}{\partial x_i} (?x?a?)i?=?xi??a?

  • 矩陣和標(biāo)量導(dǎo)數(shù)：
    (?a?X)ij=?a?Xij\left( \frac{\partial a}{\partial \boldsymbol X} \right)_{ij} = \frac{\partial a}{\partial X_{ij}} (?X?a?)ij?=?Xij??a?

  • 一階導(dǎo)數(shù)（梯度）:
    (?f(x))i=?f(x)?xi(\nabla f(\boldsymbol x))_i = \frac{\partial f(\boldsymbol x)}{\partial x_i} (?f(x))i?=?xi??f(x)?

(?2f(x))ij=?2f(x)?xi?xj(\nabla^2 f(\boldsymbol x))_{ij} = \frac{\partial^2 f(\boldsymbol x)}{\partial x_i\partial x_j} (?2f(x))ij?=?xi??xj??2f(x)?

  • 向量和向量導(dǎo)數(shù)：

    采用分子布局，即分子為列向量，分母為行向量。對(duì)分子每個(gè)元素，求導(dǎo)得一個(gè)行向量，最終組成一個(gè)矩陣。以 f(x)∈R2,x∈R3f(x)\in R^2, x\in R^3f(x)∈R2,x∈R3 為例，兩者均為列向量：

?f?x=?[f1f2]?[x1,x2,x3]=[?f1?x1,?f1?x2,?f1?x3?f2?x1,?f2?x2,?f2?x3]\frac{\partial f}{\partial x} = \frac{\partial \left[\begin{matrix} f_1 \\ f_2 \end{matrix}\right] }{\partial \left[\begin{matrix} x_1, x_2,x_3 \end{matrix}\right]}= \left[ \begin{matrix} \frac{\partial f_1}{\partial x_1}, \frac{\partial f_1}{\partial x_2} , \frac{\partial f_1}{\partial x_3} \\ \frac{\partial f_2}{\partial x_1}, \frac{\partial f_2}{\partial x_2} , \frac{\partial f_2}{\partial x_3} \end{matrix} \right] ?x?f?=?[x1?,x2?,x3??]?[f1?f2??]?=[?x1??f1??,?x2??f1??,?x3??f1???x1??f2??,?x2??f2??,?x3??f2???]

看的時(shí)候，可以把分母看成一個(gè)整體。

  • 復(fù)合求導(dǎo)

    若 x→y→fx\rightarrow y \rightarrow fx→y→f ，其中 x∈Rn,y∈R,f∈Rx\in R^n, y\in R, f\in Rx∈Rn,y∈R,f∈R ，則
    ?f?x=?f?y?y?x\frac{\partial f}{\partial x} = \frac{\partial f}{\partial y} \frac{\partial y}{\partial x} ?x?f?=?y?f??x?y?

3.2 例子1

  • f(x)=aTx=xTaf(x) = a^T x = x^T af(x)=aTx=xTa, ?f(x)=a\nabla f(x) = a?f(x)=a
  • f=tr(ATB)f = tr(A^T B)f=tr(ATB), ?Bf=A\nabla_B f = A?B?f=A, ?Af=B\nabla_A f = B?A?f=B
  • f=xTxf = x^T xf=xTx
  • f=xTAx,?xf=(A+AT)x,?x2f=2(A+AT)f = x^T A x, \nabla_x f= (A+A^T)x, \nabla_x^2 f= 2 (A+ A^T)f=xTAx,?x?f=(A+AT)x,?x2?f=2(A+AT)

3.3 微分

d(ABC)=dA?BC+A?dB?C+AB?dCdtr(X)=tr(dX){\rm d} (ABC) = {\rm d}A\cdot BC + A\cdot{\rm d}B\cdot C + AB\cdot {\rm d}C \\ {\rm d} tr(X) = tr({\rm d}X) d(ABC)=dA?BC+A?dB?C+AB?dCdtr(X)=tr(dX)

逐元素函數(shù)的微分：
dσ(x)=σ′(x)⊙dxdσ(X)=σ′(X)⊙dXd\sigma(x) = \sigma'(x)\odot dx \\ d\sigma(X) = \sigma'(X)\odot dX dσ(x)=σ′(x)⊙dxdσ(X)=σ′(X)⊙dX
其中 σ,σ′\sigma, \sigma'σ,σ′ 為逐元素函數(shù)及對(duì)應(yīng)導(dǎo)數(shù)，⊙\odot⊙ 代表逐元素相乘。

3.4 向量及矩陣求導(dǎo)

利用微分和跡求導(dǎo)。

  • 回顧：標(biāo)量對(duì)標(biāo)量求導(dǎo)：對(duì)標(biāo)量 x∈R,f(x)∈Rx\in R, f(x)\in Rx∈R,f(x)∈R，若 df=adxdf = a dxdf=adx，則 dfdx=a\frac{df}{dx} = adxdf?=a .

  • 回顧：推廣：標(biāo)量對(duì)標(biāo)量求導(dǎo)：全微分表達(dá)式 f(x,y)=x2+xy+y2f(x,y) = x^2 + xy + y^2f(x,y)=x2+xy+y2， df=2xdx+ydx+xdy+2ydy=(2x+y)dx+(x+2y)dydf = 2x dx + y dx + x dy+ 2y dy = (2x + y)dx + (x + 2y)dydf=2xdx+ydx+xdy+2ydy=(2x+y)dx+(x+2y)dy，則 ?f?x=(2x+y),?f?y=(x+2y)\frac{\partial f}{\partial x} = (2x + y), \frac{\partial f}{\partial y} = (x + 2y)?x?f?=(2x+y),?y?f?=(x+2y) .

  • 標(biāo)量對(duì)向量求導(dǎo)：對(duì)向量 x∈Rn,f(x)∈Rx\in R^n, f(x)\in Rx∈Rn,f(x)∈R，如果 df=aTdxdf = a^T dxdf=aTdx，則 dfdx=a\frac{df}{dx} = adxdf?=a 。

  • 推廣：標(biāo)量對(duì)向量求導(dǎo)：對(duì)向量 x,y∈Rn,f(x),f(y)∈Rx,y\in R^n, f(x), f(y)\in Rx,y∈Rn,f(x),f(y)∈R，如果 df=aTdx+bTdydf = a^T dx + b^T dydf=aTdx+bTdy，則 ?f?x=a,?f?y=b\frac{\partial f}{\partial x} = a, \frac{\partial f}{\partial y}=b?x?f?=a,?y?f?=b 。

  • 標(biāo)量對(duì)矩陣求導(dǎo)：對(duì)矩陣 X,Y∈Rm×n,f(X),f(Y)∈RX,Y\in R^{m\times n}, f(X), f(Y)\in RX,Y∈Rm×n,f(X),f(Y)∈R，如果 df=tr(ATdX)+tr(BTdY)df = tr(A^T dX) + tr(B^T dY)df=tr(ATdX)+tr(BTdY)，則 ?f?X=A,?f?Y=B\frac{\partial f}{\partial X}=A, \frac{\partial f}{\partial Y}=B?X?f?=A,?Y?f?=B。

  • 推廣：標(biāo)量對(duì)矩陣求導(dǎo)：對(duì)矩陣 X,Y∈Rm×n,f(X),f(Y)∈RX,Y\in R^{m\times n}, f(X), f(Y)\in RX,Y∈Rm×n,f(X),f(Y)∈R，如果 df=tr(ATdX)+tr(BTdY)df = tr(A^T dX) + tr(B^T dY)df=tr(ATdX)+tr(BTdY)，則 ?f?X=A,?f?Y=B\frac{\partial f}{\partial X}=A, \frac{\partial f}{\partial Y}=B?X?f?=A,?Y?f?=B。

3.5 矩陣求導(dǎo)在機(jī)器學(xué)習(xí)算法中的應(yīng)用

  • f(x)=∥Ax?b∥2f(x) = \Vert Ax - b\Vert^2f(x)=∥Ax?b∥2
    f(x)=∥Ax?b∥2=(Ax?b)T(Ax?b)\begin{aligned} f(x) &= \Vert Ax - b\Vert^2 \\ &= (Ax-b)^T (Ax-b) \end{aligned} f(x)?=∥Ax?b∥2=(Ax?b)T(Ax?b)?

df(x)=(Adx)T(Ax?b)+(Ax?b)TAdx=tr((2AT(Ax?b))Tdx)df(x)dx=2AT(Ax?b)df(x) = (A dx)^T(Ax - b) + (Ax-b)^T A dx = tr((2A^T (Ax -b))^T dx) \\ \frac{df(x)}{dx} = 2A^T (Ax-b) df(x)=(Adx)T(Ax?b)+(Ax?b)TAdx=tr((2AT(Ax?b))Tdx)dxdf(x)?=2AT(Ax?b)

  • f(A)=∥A∥F2f(A) = \Vert A\Vert_F^2f(A)=∥A∥F2?
    f(A)=∥A∥F2=tr(ATA)f(A) = \Vert A\Vert_F^2 = tr(A^T A) f(A)=∥A∥F2?=tr(ATA)

df(A)=dtr(ATA)=tr(dATA)+tr(ATdA)=tr(ATdA)+tr(ATdA)=tr(2ATdA)\begin{aligned} df(A) &= dtr(A^T A)\\ &= tr(dA^T A) + tr(A^T dA) \\ &= tr(A^T dA) + tr(A^T dA) \\ &= tr(2A^T dA) \end{aligned} df(A)?=dtr(ATA)=tr(dATA)+tr(ATdA)=tr(ATdA)+tr(ATdA)=tr(2ATdA)?
則 df(A)dA=2A\frac{df(A)}{dA} = 2AdAdf(A)?=2A.

  • f(P)=?tr(PTCP)+ν(PTP?I)f(P) = -tr(P^T C P) + \nu (P^T P - I)f(P)=?tr(PTCP)+ν(PTP?I) ，其中 PPP 為正交矩陣，CCC 為實(shí)對(duì)稱矩陣, ν∈R\nu\in Rν∈R 為常數(shù)。
  • f(w)=12∥w∥2+λ[y(wTx+b)?1]f(w) = \frac{1}{2} \Vert w \Vert^2 + \lambda [y(w^T x + b) -1]f(w)=21?∥w∥2+λ[y(wTx+b)?1]，其中 y,b,λ∈Ry,b,\lambda \in Ry,b,λ∈R 為常數(shù)。

4. 編程基礎(chǔ)

import numpy as np import pandas as pd import matplotlib.pyplot as plt import scipy.stats import sklearnx = np.array([1,2,3]) y = np.array([1,1,1]) A = np.random.randint(0, 9, (3,3)) B = np.random.randint(0, 9, (3,3))np.trace(A) np.linalg.norm(x, ord=np.inf) np.linalg.norm(A, ord='fro') import numpy as np import matplotlib.pyplot as plt# 1/2 范數(shù)繪制 x = np.linspace(-1, 1, 199) # 注意點(diǎn)數(shù)為奇數(shù)，否則尖點(diǎn)出不來(lái) y05 = (1 - abs(x)**(1/2) )**2plt.figure(figsize=(4,4)) plt.plot(x, y05, 'r', x, -y05, 'r') plt.xlim([-1.2, 1.2]); plt.ylim([-1.2, 1.2]) plt.axis('square') plt.axis('off') plt.show()

5. 機(jī)器學(xué)習(xí)框架

準(zhǔn)確地講，機(jī)器學(xué)習(xí)有很多類別，其中一個(gè)主要的框架如下：

#mermaid-svg-fkV974HDqQQNlTWx .label{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);fill:#333;color:#333}#mermaid-svg-fkV974HDqQQNlTWx .label text{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .node rect,#mermaid-svg-fkV974HDqQQNlTWx .node circle,#mermaid-svg-fkV974HDqQQNlTWx .node ellipse,#mermaid-svg-fkV974HDqQQNlTWx .node polygon,#mermaid-svg-fkV974HDqQQNlTWx .node path{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-fkV974HDqQQNlTWx .node .label{text-align:center;fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .node.clickable{cursor:pointer}#mermaid-svg-fkV974HDqQQNlTWx .arrowheadPath{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .edgePath .path{stroke:#333;stroke-width:1.5px}#mermaid-svg-fkV974HDqQQNlTWx .flowchart-link{stroke:#333;fill:none}#mermaid-svg-fkV974HDqQQNlTWx .edgeLabel{background-color:#e8e8e8;text-align:center}#mermaid-svg-fkV974HDqQQNlTWx .edgeLabel rect{opacity:0.9}#mermaid-svg-fkV974HDqQQNlTWx .edgeLabel span{color:#333}#mermaid-svg-fkV974HDqQQNlTWx .cluster rect{fill:#ffffde;stroke:#aa3;stroke-width:1px}#mermaid-svg-fkV974HDqQQNlTWx .cluster text{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:12px;background:#ffffde;border:1px solid #aa3;border-radius:2px;pointer-events:none;z-index:100}#mermaid-svg-fkV974HDqQQNlTWx .actor{stroke:#ccf;fill:#ECECFF}#mermaid-svg-fkV974HDqQQNlTWx text.actor>tspan{fill:#000;stroke:none}#mermaid-svg-fkV974HDqQQNlTWx .actor-line{stroke:grey}#mermaid-svg-fkV974HDqQQNlTWx .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333}#mermaid-svg-fkV974HDqQQNlTWx .messageLine1{stroke-width:1.5;stroke-dasharray:2, 2;stroke:#333}#mermaid-svg-fkV974HDqQQNlTWx #arrowhead path{fill:#333;stroke:#333}#mermaid-svg-fkV974HDqQQNlTWx .sequenceNumber{fill:#fff}#mermaid-svg-fkV974HDqQQNlTWx #sequencenumber{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx #crosshead path{fill:#333;stroke:#333}#mermaid-svg-fkV974HDqQQNlTWx .messageText{fill:#333;stroke:#333}#mermaid-svg-fkV974HDqQQNlTWx .labelBox{stroke:#ccf;fill:#ECECFF}#mermaid-svg-fkV974HDqQQNlTWx .labelText,#mermaid-svg-fkV974HDqQQNlTWx .labelText>tspan{fill:#000;stroke:none}#mermaid-svg-fkV974HDqQQNlTWx .loopText,#mermaid-svg-fkV974HDqQQNlTWx .loopText>tspan{fill:#000;stroke:none}#mermaid-svg-fkV974HDqQQNlTWx .loopLine{stroke-width:2px;stroke-dasharray:2, 2;stroke:#ccf;fill:#ccf}#mermaid-svg-fkV974HDqQQNlTWx .note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-fkV974HDqQQNlTWx .noteText,#mermaid-svg-fkV974HDqQQNlTWx .noteText>tspan{fill:#000;stroke:none}#mermaid-svg-fkV974HDqQQNlTWx .activation0{fill:#f4f4f4;stroke:#666}#mermaid-svg-fkV974HDqQQNlTWx .activation1{fill:#f4f4f4;stroke:#666}#mermaid-svg-fkV974HDqQQNlTWx .activation2{fill:#f4f4f4;stroke:#666}#mermaid-svg-fkV974HDqQQNlTWx .mermaid-main-font{font-family:"trebuchet ms", verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .section{stroke:none;opacity:0.2}#mermaid-svg-fkV974HDqQQNlTWx .section0{fill:rgba(102,102,255,0.49)}#mermaid-svg-fkV974HDqQQNlTWx .section2{fill:#fff400}#mermaid-svg-fkV974HDqQQNlTWx .section1,#mermaid-svg-fkV974HDqQQNlTWx .section3{fill:#fff;opacity:0.2}#mermaid-svg-fkV974HDqQQNlTWx .sectionTitle0{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .sectionTitle1{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .sectionTitle2{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .sectionTitle3{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .sectionTitle{text-anchor:start;font-size:11px;text-height:14px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .grid .tick{stroke:#d3d3d3;opacity:0.8;shape-rendering:crispEdges}#mermaid-svg-fkV974HDqQQNlTWx .grid .tick text{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .grid path{stroke-width:0}#mermaid-svg-fkV974HDqQQNlTWx .today{fill:none;stroke:red;stroke-width:2px}#mermaid-svg-fkV974HDqQQNlTWx .task{stroke-width:2}#mermaid-svg-fkV974HDqQQNlTWx .taskText{text-anchor:middle;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .taskText:not([font-size]){font-size:11px}#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutsideRight{fill:#000;text-anchor:start;font-size:11px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutsideLeft{fill:#000;text-anchor:end;font-size:11px}#mermaid-svg-fkV974HDqQQNlTWx .task.clickable{cursor:pointer}#mermaid-svg-fkV974HDqQQNlTWx .taskText.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutsideLeft.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutsideRight.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-fkV974HDqQQNlTWx .taskText0,#mermaid-svg-fkV974HDqQQNlTWx .taskText1,#mermaid-svg-fkV974HDqQQNlTWx .taskText2,#mermaid-svg-fkV974HDqQQNlTWx .taskText3{fill:#fff}#mermaid-svg-fkV974HDqQQNlTWx .task0,#mermaid-svg-fkV974HDqQQNlTWx .task1,#mermaid-svg-fkV974HDqQQNlTWx .task2,#mermaid-svg-fkV974HDqQQNlTWx .task3{fill:#8a90dd;stroke:#534fbc}#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutside0,#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutside2{fill:#000}#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutside1,#mermaid-svg-fkV974HDqQQNlTWx .taskTextOutside3{fill:#000}#mermaid-svg-fkV974HDqQQNlTWx .active0,#mermaid-svg-fkV974HDqQQNlTWx .active1,#mermaid-svg-fkV974HDqQQNlTWx .active2,#mermaid-svg-fkV974HDqQQNlTWx .active3{fill:#bfc7ff;stroke:#534fbc}#mermaid-svg-fkV974HDqQQNlTWx .activeText0,#mermaid-svg-fkV974HDqQQNlTWx .activeText1,#mermaid-svg-fkV974HDqQQNlTWx .activeText2,#mermaid-svg-fkV974HDqQQNlTWx .activeText3{fill:#000 !important}#mermaid-svg-fkV974HDqQQNlTWx .done0,#mermaid-svg-fkV974HDqQQNlTWx .done1,#mermaid-svg-fkV974HDqQQNlTWx .done2,#mermaid-svg-fkV974HDqQQNlTWx .done3{stroke:grey;fill:#d3d3d3;stroke-width:2}#mermaid-svg-fkV974HDqQQNlTWx .doneText0,#mermaid-svg-fkV974HDqQQNlTWx .doneText1,#mermaid-svg-fkV974HDqQQNlTWx .doneText2,#mermaid-svg-fkV974HDqQQNlTWx .doneText3{fill:#000 !important}#mermaid-svg-fkV974HDqQQNlTWx .crit0,#mermaid-svg-fkV974HDqQQNlTWx .crit1,#mermaid-svg-fkV974HDqQQNlTWx .crit2,#mermaid-svg-fkV974HDqQQNlTWx .crit3{stroke:#f88;fill:red;stroke-width:2}#mermaid-svg-fkV974HDqQQNlTWx .activeCrit0,#mermaid-svg-fkV974HDqQQNlTWx .activeCrit1,#mermaid-svg-fkV974HDqQQNlTWx .activeCrit2,#mermaid-svg-fkV974HDqQQNlTWx .activeCrit3{stroke:#f88;fill:#bfc7ff;stroke-width:2}#mermaid-svg-fkV974HDqQQNlTWx .doneCrit0,#mermaid-svg-fkV974HDqQQNlTWx .doneCrit1,#mermaid-svg-fkV974HDqQQNlTWx .doneCrit2,#mermaid-svg-fkV974HDqQQNlTWx .doneCrit3{stroke:#f88;fill:#d3d3d3;stroke-width:2;cursor:pointer;shape-rendering:crispEdges}#mermaid-svg-fkV974HDqQQNlTWx .milestone{transform:rotate(45deg) scale(0.8, 0.8)}#mermaid-svg-fkV974HDqQQNlTWx .milestoneText{font-style:italic}#mermaid-svg-fkV974HDqQQNlTWx .doneCritText0,#mermaid-svg-fkV974HDqQQNlTWx .doneCritText1,#mermaid-svg-fkV974HDqQQNlTWx .doneCritText2,#mermaid-svg-fkV974HDqQQNlTWx .doneCritText3{fill:#000 !important}#mermaid-svg-fkV974HDqQQNlTWx .activeCritText0,#mermaid-svg-fkV974HDqQQNlTWx .activeCritText1,#mermaid-svg-fkV974HDqQQNlTWx .activeCritText2,#mermaid-svg-fkV974HDqQQNlTWx .activeCritText3{fill:#000 !important}#mermaid-svg-fkV974HDqQQNlTWx .titleText{text-anchor:middle;font-size:18px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx g.classGroup text{fill:#9370db;stroke:none;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:10px}#mermaid-svg-fkV974HDqQQNlTWx g.classGroup text .title{font-weight:bolder}#mermaid-svg-fkV974HDqQQNlTWx g.clickable{cursor:pointer}#mermaid-svg-fkV974HDqQQNlTWx g.classGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-fkV974HDqQQNlTWx g.classGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5}#mermaid-svg-fkV974HDqQQNlTWx .classLabel .label{fill:#9370db;font-size:10px}#mermaid-svg-fkV974HDqQQNlTWx .relation{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-fkV974HDqQQNlTWx .dashed-line{stroke-dasharray:3}#mermaid-svg-fkV974HDqQQNlTWx #compositionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #compositionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #aggregationStart{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #aggregationEnd{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #dependencyStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #dependencyEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #extensionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx #extensionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx .commit-id,#mermaid-svg-fkV974HDqQQNlTWx .commit-msg,#mermaid-svg-fkV974HDqQQNlTWx .branch-label{fill:lightgrey;color:lightgrey;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .pieTitleText{text-anchor:middle;font-size:25px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .slice{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx g.stateGroup text{fill:#9370db;stroke:none;font-size:10px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx g.stateGroup text{fill:#9370db;fill:#333;stroke:none;font-size:10px}#mermaid-svg-fkV974HDqQQNlTWx g.statediagram-cluster .cluster-label text{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx g.stateGroup .state-title{font-weight:bolder;fill:#000}#mermaid-svg-fkV974HDqQQNlTWx g.stateGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-fkV974HDqQQNlTWx g.stateGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-fkV974HDqQQNlTWx .transition{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-fkV974HDqQQNlTWx .stateGroup .composit{fill:white;border-bottom:1px}#mermaid-svg-fkV974HDqQQNlTWx .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px}#mermaid-svg-fkV974HDqQQNlTWx .state-note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-fkV974HDqQQNlTWx .state-note text{fill:black;stroke:none;font-size:10px}#mermaid-svg-fkV974HDqQQNlTWx .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.7}#mermaid-svg-fkV974HDqQQNlTWx .edgeLabel text{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .stateLabel text{fill:#000;font-size:10px;font-weight:bold;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-fkV974HDqQQNlTWx .node circle.state-start{fill:black;stroke:black}#mermaid-svg-fkV974HDqQQNlTWx .node circle.state-end{fill:black;stroke:white;stroke-width:1.5}#mermaid-svg-fkV974HDqQQNlTWx #statediagram-barbEnd{fill:#9370db}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-cluster rect{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-cluster rect.outer{rx:5px;ry:5px}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-state .divider{stroke:#9370db}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-state .title-state{rx:5px;ry:5px}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-cluster.statediagram-cluster .inner{fill:white}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-cluster.statediagram-cluster-alt .inner{fill:#e0e0e0}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-cluster .inner{rx:0;ry:0}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-state rect.basic{rx:5px;ry:5px}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#efefef}#mermaid-svg-fkV974HDqQQNlTWx .note-edge{stroke-dasharray:5}#mermaid-svg-fkV974HDqQQNlTWx .statediagram-note rect{fill:#fff5ad;stroke:#aa3;stroke-width:1px;rx:0;ry:0}:root{--mermaid-font-family: '"trebuchet ms", verdana, arial';--mermaid-font-family: "Comic Sans MS", "Comic Sans", cursive}#mermaid-svg-fkV974HDqQQNlTWx .error-icon{fill:#522}#mermaid-svg-fkV974HDqQQNlTWx .error-text{fill:#522;stroke:#522}#mermaid-svg-fkV974HDqQQNlTWx .edge-thickness-normal{stroke-width:2px}#mermaid-svg-fkV974HDqQQNlTWx .edge-thickness-thick{stroke-width:3.5px}#mermaid-svg-fkV974HDqQQNlTWx .edge-pattern-solid{stroke-dasharray:0}#mermaid-svg-fkV974HDqQQNlTWx .edge-pattern-dashed{stroke-dasharray:3}#mermaid-svg-fkV974HDqQQNlTWx .edge-pattern-dotted{stroke-dasharray:2}#mermaid-svg-fkV974HDqQQNlTWx .marker{fill:#333}#mermaid-svg-fkV974HDqQQNlTWx .marker.cross{stroke:#333}:root { --mermaid-font-family: "trebuchet ms", verdana, arial;}#mermaid-svg-fkV974HDqQQNlTWx {color: rgba(0, 0, 0, 0.75);font: ;}數(shù)據(jù)+預(yù)處理數(shù)學(xué)建模模型求解

以波士頓房?jī)r(jià)預(yù)測(cè)為例，使用線性回歸模型：

#mermaid-svg-LU7TCkpG3ANH3H9C .label{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);fill:#333;color:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .label text{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .node rect,#mermaid-svg-LU7TCkpG3ANH3H9C .node circle,#mermaid-svg-LU7TCkpG3ANH3H9C .node ellipse,#mermaid-svg-LU7TCkpG3ANH3H9C .node polygon,#mermaid-svg-LU7TCkpG3ANH3H9C .node path{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-LU7TCkpG3ANH3H9C .node .label{text-align:center;fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .node.clickable{cursor:pointer}#mermaid-svg-LU7TCkpG3ANH3H9C .arrowheadPath{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .edgePath .path{stroke:#333;stroke-width:1.5px}#mermaid-svg-LU7TCkpG3ANH3H9C .flowchart-link{stroke:#333;fill:none}#mermaid-svg-LU7TCkpG3ANH3H9C .edgeLabel{background-color:#e8e8e8;text-align:center}#mermaid-svg-LU7TCkpG3ANH3H9C .edgeLabel rect{opacity:0.9}#mermaid-svg-LU7TCkpG3ANH3H9C .edgeLabel span{color:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .cluster rect{fill:#ffffde;stroke:#aa3;stroke-width:1px}#mermaid-svg-LU7TCkpG3ANH3H9C .cluster text{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:12px;background:#ffffde;border:1px solid #aa3;border-radius:2px;pointer-events:none;z-index:100}#mermaid-svg-LU7TCkpG3ANH3H9C .actor{stroke:#ccf;fill:#ECECFF}#mermaid-svg-LU7TCkpG3ANH3H9C text.actor>tspan{fill:#000;stroke:none}#mermaid-svg-LU7TCkpG3ANH3H9C .actor-line{stroke:grey}#mermaid-svg-LU7TCkpG3ANH3H9C .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .messageLine1{stroke-width:1.5;stroke-dasharray:2, 2;stroke:#333}#mermaid-svg-LU7TCkpG3ANH3H9C #arrowhead path{fill:#333;stroke:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .sequenceNumber{fill:#fff}#mermaid-svg-LU7TCkpG3ANH3H9C #sequencenumber{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C #crosshead path{fill:#333;stroke:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .messageText{fill:#333;stroke:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .labelBox{stroke:#ccf;fill:#ECECFF}#mermaid-svg-LU7TCkpG3ANH3H9C .labelText,#mermaid-svg-LU7TCkpG3ANH3H9C .labelText>tspan{fill:#000;stroke:none}#mermaid-svg-LU7TCkpG3ANH3H9C .loopText,#mermaid-svg-LU7TCkpG3ANH3H9C .loopText>tspan{fill:#000;stroke:none}#mermaid-svg-LU7TCkpG3ANH3H9C .loopLine{stroke-width:2px;stroke-dasharray:2, 2;stroke:#ccf;fill:#ccf}#mermaid-svg-LU7TCkpG3ANH3H9C .note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-LU7TCkpG3ANH3H9C .noteText,#mermaid-svg-LU7TCkpG3ANH3H9C .noteText>tspan{fill:#000;stroke:none}#mermaid-svg-LU7TCkpG3ANH3H9C .activation0{fill:#f4f4f4;stroke:#666}#mermaid-svg-LU7TCkpG3ANH3H9C .activation1{fill:#f4f4f4;stroke:#666}#mermaid-svg-LU7TCkpG3ANH3H9C .activation2{fill:#f4f4f4;stroke:#666}#mermaid-svg-LU7TCkpG3ANH3H9C .mermaid-main-font{font-family:"trebuchet ms", verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .section{stroke:none;opacity:0.2}#mermaid-svg-LU7TCkpG3ANH3H9C .section0{fill:rgba(102,102,255,0.49)}#mermaid-svg-LU7TCkpG3ANH3H9C .section2{fill:#fff400}#mermaid-svg-LU7TCkpG3ANH3H9C .section1,#mermaid-svg-LU7TCkpG3ANH3H9C .section3{fill:#fff;opacity:0.2}#mermaid-svg-LU7TCkpG3ANH3H9C .sectionTitle0{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .sectionTitle1{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .sectionTitle2{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .sectionTitle3{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .sectionTitle{text-anchor:start;font-size:11px;text-height:14px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .grid .tick{stroke:#d3d3d3;opacity:0.8;shape-rendering:crispEdges}#mermaid-svg-LU7TCkpG3ANH3H9C .grid .tick text{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .grid path{stroke-width:0}#mermaid-svg-LU7TCkpG3ANH3H9C .today{fill:none;stroke:red;stroke-width:2px}#mermaid-svg-LU7TCkpG3ANH3H9C .task{stroke-width:2}#mermaid-svg-LU7TCkpG3ANH3H9C .taskText{text-anchor:middle;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .taskText:not([font-size]){font-size:11px}#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutsideRight{fill:#000;text-anchor:start;font-size:11px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutsideLeft{fill:#000;text-anchor:end;font-size:11px}#mermaid-svg-LU7TCkpG3ANH3H9C .task.clickable{cursor:pointer}#mermaid-svg-LU7TCkpG3ANH3H9C .taskText.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutsideLeft.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutsideRight.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-LU7TCkpG3ANH3H9C .taskText0,#mermaid-svg-LU7TCkpG3ANH3H9C .taskText1,#mermaid-svg-LU7TCkpG3ANH3H9C .taskText2,#mermaid-svg-LU7TCkpG3ANH3H9C .taskText3{fill:#fff}#mermaid-svg-LU7TCkpG3ANH3H9C .task0,#mermaid-svg-LU7TCkpG3ANH3H9C .task1,#mermaid-svg-LU7TCkpG3ANH3H9C .task2,#mermaid-svg-LU7TCkpG3ANH3H9C .task3{fill:#8a90dd;stroke:#534fbc}#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutside0,#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutside2{fill:#000}#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutside1,#mermaid-svg-LU7TCkpG3ANH3H9C .taskTextOutside3{fill:#000}#mermaid-svg-LU7TCkpG3ANH3H9C .active0,#mermaid-svg-LU7TCkpG3ANH3H9C .active1,#mermaid-svg-LU7TCkpG3ANH3H9C .active2,#mermaid-svg-LU7TCkpG3ANH3H9C .active3{fill:#bfc7ff;stroke:#534fbc}#mermaid-svg-LU7TCkpG3ANH3H9C .activeText0,#mermaid-svg-LU7TCkpG3ANH3H9C .activeText1,#mermaid-svg-LU7TCkpG3ANH3H9C .activeText2,#mermaid-svg-LU7TCkpG3ANH3H9C .activeText3{fill:#000 !important}#mermaid-svg-LU7TCkpG3ANH3H9C .done0,#mermaid-svg-LU7TCkpG3ANH3H9C .done1,#mermaid-svg-LU7TCkpG3ANH3H9C .done2,#mermaid-svg-LU7TCkpG3ANH3H9C .done3{stroke:grey;fill:#d3d3d3;stroke-width:2}#mermaid-svg-LU7TCkpG3ANH3H9C .doneText0,#mermaid-svg-LU7TCkpG3ANH3H9C .doneText1,#mermaid-svg-LU7TCkpG3ANH3H9C .doneText2,#mermaid-svg-LU7TCkpG3ANH3H9C .doneText3{fill:#000 !important}#mermaid-svg-LU7TCkpG3ANH3H9C .crit0,#mermaid-svg-LU7TCkpG3ANH3H9C .crit1,#mermaid-svg-LU7TCkpG3ANH3H9C .crit2,#mermaid-svg-LU7TCkpG3ANH3H9C .crit3{stroke:#f88;fill:red;stroke-width:2}#mermaid-svg-LU7TCkpG3ANH3H9C .activeCrit0,#mermaid-svg-LU7TCkpG3ANH3H9C .activeCrit1,#mermaid-svg-LU7TCkpG3ANH3H9C .activeCrit2,#mermaid-svg-LU7TCkpG3ANH3H9C .activeCrit3{stroke:#f88;fill:#bfc7ff;stroke-width:2}#mermaid-svg-LU7TCkpG3ANH3H9C .doneCrit0,#mermaid-svg-LU7TCkpG3ANH3H9C .doneCrit1,#mermaid-svg-LU7TCkpG3ANH3H9C .doneCrit2,#mermaid-svg-LU7TCkpG3ANH3H9C .doneCrit3{stroke:#f88;fill:#d3d3d3;stroke-width:2;cursor:pointer;shape-rendering:crispEdges}#mermaid-svg-LU7TCkpG3ANH3H9C .milestone{transform:rotate(45deg) scale(0.8, 0.8)}#mermaid-svg-LU7TCkpG3ANH3H9C .milestoneText{font-style:italic}#mermaid-svg-LU7TCkpG3ANH3H9C .doneCritText0,#mermaid-svg-LU7TCkpG3ANH3H9C .doneCritText1,#mermaid-svg-LU7TCkpG3ANH3H9C .doneCritText2,#mermaid-svg-LU7TCkpG3ANH3H9C .doneCritText3{fill:#000 !important}#mermaid-svg-LU7TCkpG3ANH3H9C .activeCritText0,#mermaid-svg-LU7TCkpG3ANH3H9C .activeCritText1,#mermaid-svg-LU7TCkpG3ANH3H9C .activeCritText2,#mermaid-svg-LU7TCkpG3ANH3H9C .activeCritText3{fill:#000 !important}#mermaid-svg-LU7TCkpG3ANH3H9C .titleText{text-anchor:middle;font-size:18px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C g.classGroup text{fill:#9370db;stroke:none;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:10px}#mermaid-svg-LU7TCkpG3ANH3H9C g.classGroup text .title{font-weight:bolder}#mermaid-svg-LU7TCkpG3ANH3H9C g.clickable{cursor:pointer}#mermaid-svg-LU7TCkpG3ANH3H9C g.classGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-LU7TCkpG3ANH3H9C g.classGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5}#mermaid-svg-LU7TCkpG3ANH3H9C .classLabel .label{fill:#9370db;font-size:10px}#mermaid-svg-LU7TCkpG3ANH3H9C .relation{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-LU7TCkpG3ANH3H9C .dashed-line{stroke-dasharray:3}#mermaid-svg-LU7TCkpG3ANH3H9C #compositionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #compositionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #aggregationStart{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #aggregationEnd{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #dependencyStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #dependencyEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #extensionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C #extensionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C .commit-id,#mermaid-svg-LU7TCkpG3ANH3H9C .commit-msg,#mermaid-svg-LU7TCkpG3ANH3H9C .branch-label{fill:lightgrey;color:lightgrey;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .pieTitleText{text-anchor:middle;font-size:25px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .slice{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C g.stateGroup text{fill:#9370db;stroke:none;font-size:10px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C g.stateGroup text{fill:#9370db;fill:#333;stroke:none;font-size:10px}#mermaid-svg-LU7TCkpG3ANH3H9C g.statediagram-cluster .cluster-label text{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C g.stateGroup .state-title{font-weight:bolder;fill:#000}#mermaid-svg-LU7TCkpG3ANH3H9C g.stateGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-LU7TCkpG3ANH3H9C g.stateGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-LU7TCkpG3ANH3H9C .transition{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-LU7TCkpG3ANH3H9C .stateGroup .composit{fill:white;border-bottom:1px}#mermaid-svg-LU7TCkpG3ANH3H9C .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px}#mermaid-svg-LU7TCkpG3ANH3H9C .state-note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-LU7TCkpG3ANH3H9C .state-note text{fill:black;stroke:none;font-size:10px}#mermaid-svg-LU7TCkpG3ANH3H9C .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.7}#mermaid-svg-LU7TCkpG3ANH3H9C .edgeLabel text{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .stateLabel text{fill:#000;font-size:10px;font-weight:bold;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-LU7TCkpG3ANH3H9C .node circle.state-start{fill:black;stroke:black}#mermaid-svg-LU7TCkpG3ANH3H9C .node circle.state-end{fill:black;stroke:white;stroke-width:1.5}#mermaid-svg-LU7TCkpG3ANH3H9C #statediagram-barbEnd{fill:#9370db}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-cluster rect{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-cluster rect.outer{rx:5px;ry:5px}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-state .divider{stroke:#9370db}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-state .title-state{rx:5px;ry:5px}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-cluster.statediagram-cluster .inner{fill:white}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-cluster.statediagram-cluster-alt .inner{fill:#e0e0e0}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-cluster .inner{rx:0;ry:0}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-state rect.basic{rx:5px;ry:5px}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#efefef}#mermaid-svg-LU7TCkpG3ANH3H9C .note-edge{stroke-dasharray:5}#mermaid-svg-LU7TCkpG3ANH3H9C .statediagram-note rect{fill:#fff5ad;stroke:#aa3;stroke-width:1px;rx:0;ry:0}:root{--mermaid-font-family: '"trebuchet ms", verdana, arial';--mermaid-font-family: "Comic Sans MS", "Comic Sans", cursive}#mermaid-svg-LU7TCkpG3ANH3H9C .error-icon{fill:#522}#mermaid-svg-LU7TCkpG3ANH3H9C .error-text{fill:#522;stroke:#522}#mermaid-svg-LU7TCkpG3ANH3H9C .edge-thickness-normal{stroke-width:2px}#mermaid-svg-LU7TCkpG3ANH3H9C .edge-thickness-thick{stroke-width:3.5px}#mermaid-svg-LU7TCkpG3ANH3H9C .edge-pattern-solid{stroke-dasharray:0}#mermaid-svg-LU7TCkpG3ANH3H9C .edge-pattern-dashed{stroke-dasharray:3}#mermaid-svg-LU7TCkpG3ANH3H9C .edge-pattern-dotted{stroke-dasharray:2}#mermaid-svg-LU7TCkpG3ANH3H9C .marker{fill:#333}#mermaid-svg-LU7TCkpG3ANH3H9C .marker.cross{stroke:#333}:root { --mermaid-font-family: "trebuchet ms", verdana, arial;}#mermaid-svg-LU7TCkpG3ANH3H9C {color: rgba(0, 0, 0, 0.75);font: ;}房?jī)r(jià)預(yù)測(cè)數(shù)據(jù)線性回歸直接求解

6. 機(jī)器學(xué)習(xí)中的最優(yōu)化問(wèn)題

6.1 建立數(shù)學(xué)模型

根據(jù)數(shù)據(jù)或者現(xiàn)實(shí)物理關(guān)系，可以建立各種各樣的數(shù)學(xué)模型。

  • 部分問(wèn)題沒(méi)有約束條件，稱為無(wú)約束問(wèn)題，例如：

    • 線性回歸

      minimizew∥y?Xw∥22\mathop{\rm minimize}\limits_w \quad \Vert y - Xw\Vert_2^2 wminimize?∥y?Xw∥22?

    • 嶺回歸
      minimizew∥y?Xw∥22+λ∥w∥22\mathop{\rm minimize}\limits_w \quad \Vert y - Xw\Vert_2^2 + \lambda \Vert w\Vert_2^2\\ wminimize?∥y?Xw∥22?+λ∥w∥22?

    • LASSO算法
      minimizew∥y?Xw∥22+λ∥w∥1\mathop{\rm minimize}\limits_w \quad \Vert y - Xw\Vert_2^2 + \lambda \Vert w\Vert_1\\ wminimize?∥y?Xw∥22?+λ∥w∥1?

    • Logistic回歸

    • BP前饋神經(jīng)網(wǎng)絡(luò)

  • 部分問(wèn)題有等式約束或不等式約束，稱為有約束問(wèn)題，例如：

    • 線性判別分析（LDA）
      minimizew?wTSbwsubjecttowTSww=c\begin{aligned} \mathop{minimize}\limits_w &\quad -w^T S_b w \\ {\rm subject\ to} &\quad w^T S_w w = c \end{aligned} wminimize?subject?to??wTSb?wwTSw?w=c?

    • 支持向量機(jī)（SVM）
      minimizew12∥w∥2subjecttoyi(wTxi+b)≤1,i=1,...,m\begin{aligned} \mathop{\rm minimize}\limits_w &\quad \frac{1}{2}\Vert w\Vert^2 \\ {\rm subject\ to} &\quad y_i(w^T x_i + b) \le 1, \quad i=1,...,m \end{aligned} wminimize?subject?to?21?∥w∥2yi?(wTxi?+b)≤1,i=1,...,m?

    • 主成分分析（PCA）
      minimizeP∑i=1m∥PPTxi?xi∥22subjecttoPTP=I\begin{aligned} \mathop{\rm minimize}\limits_P &\quad \sum_{i=1}^m \Vert PP^T x_i - x_i\Vert_2^2 \\ {\rm subject\ to} &\quad P^TP = I \end{aligned} Pminimize?subject?to?i=1∑m?∥PPTxi??xi?∥22?PTP=I?

6.2 求解優(yōu)化變量

機(jī)器學(xué)習(xí)的任務(wù)：

  • 高端說(shuō)法：從數(shù)據(jù)中獲取知識(shí)，賦予機(jī)器智慧（高端說(shuō)法）。
  • 接地氣說(shuō)法：通過(guò)調(diào)整參數(shù)（權(quán)重），使目標(biāo)損失函數(shù)最小化。

    • 求解參數(shù)方法：

    • 直接求解。對(duì)于線性回歸和嶺回歸，可以直接求出優(yōu)化變量的表達(dá)式；
    • 迭代法。對(duì)于一些凸函數(shù)，直接求出優(yōu)化變量表達(dá)式很困難，或者計(jì)算過(guò)于復(fù)雜，通常會(huì)采取迭代方法求解，例如 梯度下降法、牛頓法，以及后面會(huì)學(xué)習(xí)的共軛梯度法等等。
    版權(quán)申明：本教程版權(quán)歸創(chuàng)作人所有，未經(jīng)許可，謝絕轉(zhuǎn)載！

    交流討論QQ群：784117704

    部分視頻觀看地址：b站搜索“火力教育”

    課件下載地址：QQ群文件（有最新更新） or 百度網(wǎng)盤(pán)PDF課件及代碼

    鏈接：https://pan.baidu.com/s/1lc8c7yDc30KY1L_ehJAfDg
    提取碼：u3ls

    總結(jié)

    以上是生活随笔為你收集整理的第1章 数学基础和机器学习问题（范数+矩阵迹+矩阵求导+机器学习框架）的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

    如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。