36函數(shù)極小值優(yōu)化實戰(zhàn)

四個解都是0所以都是全局最小值


PLOT

#最小值0取的值（x,y）是（3,2）x初始化是0 #最小值0取的值（x,y）是（3.584428，-1.8484=126）x初始化是4import os os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'import torch import numpy as np import matplotlib.pyplot as plt from mpl_toolkits.mplot3d import Axes3D #f(x,y)=(x**2+y-11)**2+(x+y**2-7)**2def f(x):return (x[0]**2+x[1]-11)**2+(x[0]+x[1]**2-7)**2x=np.arange(-6,6,0.1) y=np.arange(-6,6,0.1) print('x,y range:',x.shape,y.shape) X,Y=np.meshgrid(x,y) print('X,Y maps:',X.shape,Y.shape)#x,y兩個平面對應(yīng)位置上的點一一對應(yīng)組成map Z=f([X,Y])fig=plt.figure('f') ax=fig.gca(projection='3d') ax.plot_surface(X,Y,Z) ax.view_init(60,-30) ax.set_xlabel('x') ax.set_ylabel('y') plt.show()#現(xiàn)在是優(yōu)化 pred對x的偏導(dǎo) 不是loss對w的偏導(dǎo)x=torch.tensor([0.,0.],requires_grad=True) optimizer=torch.optim.Adam([x],lr=1e-3) #完成對x[0]'=x[0]-lr*pred對x[0]的偏導(dǎo) #完成對x[1]'=x[1]-lr*pred對x[1]的偏導(dǎo)for step in range(20000):pred=f(x)optimizer.zero_grad()#反向傳播累乘前先清零pred.backward()#生成x[0] x[1]的梯度信息optimizer.step()#更新一次x[0]' x[1]' 直到找到x[0]* x[1]*最合適的 滿足最小值if step%2000==0:print('step {} ：x= {}，f(x)={}'.format(step,x.tolist(),pred.item())) ''' x,y range: (120,) (120,) X,Y maps: (120, 120) (120, 120) step 0 ：x= [0.0009999999310821295, 0.0009999999310821295]，f(x)=170.0 step 2000 ：x= [2.3331806659698486, 1.9540694952011108]，f(x)=13.730916023254395 step 4000 ：x= [2.9820079803466797, 2.0270984172821045]，f(x)=0.014858869835734367 step 6000 ：x= [2.999983549118042, 2.0000221729278564]，f(x)=1.1074007488787174e-08 step 8000 ：x= [2.9999938011169434, 2.0000083446502686]，f(x)=1.5572823031106964e-09 step 10000 ：x= [2.999997854232788, 2.000002861022949]，f(x)=1.8189894035458565e-10 step 12000 ：x= [2.9999992847442627, 2.0000009536743164]，f(x)=1.6370904631912708e-11 step 14000 ：x= [2.999999761581421, 2.000000238418579]，f(x)=1.8189894035458565e-12 step 16000 ：x= [3.0, 2.0]，f(x)=0.0 step 18000 ：x= [3.0, 2.0]，f(x)=0.0Process finished with exit code 0'''

37logistics regression

regression一般是指的連續(xù)的值
轉(zhuǎn)換成分類問題 加個sigmoid函數(shù)

classification：
目標(biāo)：放大accuracy 越準(zhǔn)確越好
pθ(y|x) 給出x得到y(tǒng)的條件概率,模型學(xué)到的概率
pr(y|x) 給出x得到y(tǒng)的條件概率,真實的概率
現(xiàn)在要法1縮小這二者的差距或者法2縮小二者的分布

為什么不能直接放大acc
logistics regression：叫l(wèi)ogistics是因為用了sigmoid函數(shù)，regression其實有爭議，其實本質(zhì)上是個classification問題 ，當(dāng)使用MSE時可以理解成regression，但是用cross entropy就是classification問題

多分類問題
經(jīng)過softmax激活后，原來的輸出大的變得更大
2->0.7，1->0.2 本來2/1=2倍現(xiàn)在0.7/0.2=3.5倍

38交叉熵

cross entropy

entropy
不確定性 驚喜 信息少

最后一個最穩(wěn)定 所以熵很低，0.999的那個中獎
第一個熵最高不穩(wěn)定會隨機一個人中獎，都是0.25
熵最小，這個時候最有秩序；而被打亂的時候，熵開始增大，直到最后一片混亂，熵變成最大：

Dkl是kl Divergence 兩個分布的重疊程度，數(shù)值越大重疊越少，數(shù)值越小重疊越大，如果pq完全相似則，Dkl就是0。

p=q時，Dkl=0，再有如果采用one-hot（分類問題多用這個），這樣entropy=1log1=0，所以越穩(wěn)定，所以可以用cross entropy衡量出來時

二分類問題的優(yōu)化目標(biāo)是-（ylog(p)+(1-y)log(1-p)）

使用cross entropy優(yōu)化H下降速度更快，優(yōu)化更快，pθ（預(yù)測分布）更接近pr（真實分布），比使用MSE更快

為啥不用MSE
1梯度消失
2梯度下降慢
3但是有時候用MSE效果更好 因為MSE梯度求導(dǎo)簡單
（pred-y）**2 求導(dǎo)得2（pred-y）

注意cross entropy在pytorch中已經(jīng)把softmax和log打包在一起，所以要傳入的是logits
ce=softmax+log+nll_loss

39多分類問題實戰(zhàn)

step是訓(xùn)練完一個batch的step
epoch是訓(xùn)練完整個數(shù)據(jù)集

''' Train Epoch：0 [0/60000 (0%)] Loss:2.496092 Train Epoch：0 [20000/60000 (33%)] Loss:1.293757 Train Epoch：0 [40000/60000 (67%)] Loss:0.846965 Train Epoch：1 [0/60000 (0%)] Loss:0.627197 Train Epoch：1 [20000/60000 (33%)] Loss:0.580098 Train Epoch：1 [40000/60000 (67%)] Loss:0.486529 Train Epoch：2 [0/60000 (0%)] Loss:0.495914 Train Epoch：2 [20000/60000 (33%)] Loss:0.588657 Train Epoch：2 [40000/60000 (67%)] Loss:0.446819 Train Epoch：3 [0/60000 (0%)] Loss:0.447987 Train Epoch：3 [20000/60000 (33%)] Loss:0.384017 Train Epoch：3 [40000/60000 (67%)] Loss:0.367968 Train Epoch：4 [0/60000 (0%)] Loss:0.477522 Train Epoch：4 [20000/60000 (33%)] Loss:0.592025 Train Epoch：4 [40000/60000 (67%)] Loss:0.540704 Train Epoch：5 [0/60000 (0%)] Loss:0.448472 Train Epoch：5 [20000/60000 (33%)] Loss:0.445303 Train Epoch：5 [40000/60000 (67%)] Loss:0.489408 Train Epoch：6 [0/60000 (0%)] Loss:0.352725 Train Epoch：6 [20000/60000 (33%)] Loss:0.431580 Train Epoch：6 [40000/60000 (67%)] Loss:0.474489 Train Epoch：7 [0/60000 (0%)] Loss:0.550202 Train Epoch：7 [20000/60000 (33%)] Loss:0.415313 Train Epoch：7 [40000/60000 (67%)] Loss:0.369599 Train Epoch：8 [0/60000 (0%)] Loss:0.500460 Train Epoch：8 [20000/60000 (33%)] Loss:0.388243 Train Epoch：8 [40000/60000 (67%)] Loss:0.404500 Train Epoch：9 [0/60000 (0%)] Loss:0.397505 Train Epoch：9 [20000/60000 (33%)] Loss:0.351531 Train Epoch：9 [40000/60000 (67%)] Loss:0.443983test set:average loss：0.0003,Accuracy:8606/10000 (86)Process finished with exit code 0'''

40全連接層

多層的全連接層
nn.Linear

layer=nn.Linear(784,200)

大部分情況用relu 像素重建的時候才用sigmoid

autograd后實行自動向后求導(dǎo)

nn.Sequenltial可以添加任何繼承nn.Module的類

import torch import torch.nn as nnclass MLP(nn.Module):def __init__(self):super(MLP,self).__init__()self.model=nn.Sequential(#sequential串聯(lián)起來nn.Linear(784,200),nn.ReLU(inplace=True),nn.Linear(200, 200),nn.ReLU(inplace=True),nn.Linear(200,10),nn.ReLU(inplace=True),)def forward(self,x):x=self.model(x)return x

nn.ReLu是類api，需要先實例化再調(diào)用，w，b是內(nèi)部參數(shù)不能介入只能使用.parameter來訪問而不是私自訪問
F.relu，function-api，可以自己管理w，b這些tensor，提供gpu操作
一般優(yōu)先nn.ReLu

這里不用初始化w和b 因為w和b已經(jīng)被Linear管理了，沒辦法直接暴露出來初始化，這個接口有自己的初始化方法已經(jīng)夠用了

# -*- codeing = utf-8 -*- # @Time :2021/5/14 21:06 # @Author:sueong # @File:ll.py # @Software:PyCharm import torch import torch.nn as nn from torch import optim# 超參數(shù) from torchvision import datasets, transformsbatch_size = 200 learning_rate = 0.01 epochs = 10# 獲取訓(xùn)練數(shù)據(jù) train_db = datasets.MNIST('../data', train=True, download=True, # train=True則得到的是訓(xùn)練集transform=transforms.Compose([ # transform進行數(shù)據(jù)預(yù)處理transforms.ToTensor(), # 轉(zhuǎn)成Tensor類型的數(shù)據(jù)transforms.Normalize((0.1307,), (0.3081,)) # 進行數(shù)據(jù)標(biāo)準(zhǔn)化(減去均值除以方差)]))# DataLoader把訓(xùn)練數(shù)據(jù)分成多個小組，此函數(shù)每次拋出一組數(shù)據(jù)。直至把所有的數(shù)據(jù)都拋出。就是做一個數(shù)據(jù)的初始化 train_loader = torch.utils.data.DataLoader(train_db, batch_size=batch_size, shuffle=True)# 獲取測試數(shù)據(jù) test_db = datasets.MNIST('../data', train=False,transform=transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.1307,), (0.3081,))]))test_loader = torch.utils.data.DataLoader(test_db, batch_size=batch_size, shuffle=True)class MLP(nn.Module):def __init__(self):super(MLP,self).__init__()self.model=nn.Sequential(#sequential串聯(lián)起來nn.Linear(784,200),nn.ReLU(inplace=True),nn.Linear(200, 200),nn.ReLU(inplace=True),nn.Linear(200,10),nn.ReLU(inplace=True),)def forward(self,x):x = self.model(x)return x#Trainnet=MLP()#網(wǎng)絡(luò)結(jié)構(gòu) 就是foward函數(shù) optimizer=optim.SGD(net.parameters(),lr=learning_rate)#使用nn.Module可以直接代替之前[w1,b1,w2,b2.。。] criteon=nn.CrossEntropyLoss()for epoch in range(epochs):for batch_ind,(data,target) in enumerate(train_loader):data=data.view(-1,28*28)logits=net(data)#這不要再加softmax logits就是predloss=criteon(logits,target)#求lossoptimizer.zero_grad()loss.backward()optimizer.step()if batch_ind%100==0:print('Train Epoch：{} [{}/{} ({:.0f}%)]\t Loss:{:.6f}'.format(epoch,batch_ind*len(data),len(train_loader.dataset),100.* batch_ind/len(train_loader),loss.item()))test_loss=0 correct=0for data,target in test_loader:data=data.view(-1,28*28)#第一個維度保持不變寫-1logits=net(data)test_loss+=criteon(logits,target).item()pred=logits.data.max(1)[1]#因為correct+=pred.eq(target.data).sum()test_loss/=len(train_loader.dataset) print('\n test set:average loss：{:.4f},Accuracy:{}/{} ({:.0f}%)\n'.format(test_loss,correct,len(test_loader.dataset),100.*correct/len(test_loader.dataset) ))''' F:\anaconda\envs\pytorch\python.exe F:/pythonProject1/pythonProject3/ll.py Train Epoch：0 [0/60000 (0%)] Loss:2.315060 Train Epoch：0 [20000/60000 (33%)] Loss:2.069264 Train Epoch：0 [40000/60000 (67%)] Loss:1.572258 Train Epoch：1 [0/60000 (0%)] Loss:1.299846 Train Epoch：1 [20000/60000 (33%)] Loss:1.100520 Train Epoch：1 [40000/60000 (67%)] Loss:1.019639 Train Epoch：2 [0/60000 (0%)] Loss:0.945382 Train Epoch：2 [20000/60000 (33%)] Loss:0.895289 Train Epoch：2 [40000/60000 (67%)] Loss:0.877365 Train Epoch：3 [0/60000 (0%)] Loss:0.826702 Train Epoch：3 [20000/60000 (33%)] Loss:0.817137 Train Epoch：3 [40000/60000 (67%)] Loss:0.667326 Train Epoch：4 [0/60000 (0%)] Loss:0.688401 Train Epoch：4 [20000/60000 (33%)] Loss:0.692957 Train Epoch：4 [40000/60000 (67%)] Loss:0.569227 Train Epoch：5 [0/60000 (0%)] Loss:0.667585 Train Epoch：5 [20000/60000 (33%)] Loss:0.588931 Train Epoch：5 [40000/60000 (67%)] Loss:0.531139 Train Epoch：6 [0/60000 (0%)] Loss:0.632487 Train Epoch：6 [20000/60000 (33%)] Loss:0.499388 Train Epoch：6 [40000/60000 (67%)] Loss:0.691728 Train Epoch：7 [0/60000 (0%)] Loss:0.435568 Train Epoch：7 [20000/60000 (33%)] Loss:0.600809 Train Epoch：7 [40000/60000 (67%)] Loss:0.525673 Train Epoch：8 [0/60000 (0%)] Loss:0.519194 Train Epoch：8 [20000/60000 (33%)] Loss:0.540513 Train Epoch：8 [40000/60000 (67%)] Loss:0.438987 Train Epoch：9 [0/60000 (0%)] Loss:0.427629 Train Epoch：9 [20000/60000 (33%)] Loss:0.409045 Train Epoch：9 [40000/60000 (67%)] Loss:0.514155test set:average loss：0.0004,Accuracy:8286/10000 (83%)Process finished with exit code 0''' 超強干貨來襲 云風(fēng)專訪：近40年碼齡，通宵達旦的技術(shù)人生

總結(jié)

以上是生活随笔為你收集整理的pytorch教程龙曲良36-40的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。