【神经网络】(16) MobileNetV3 代码复现，网络解析，附Tensorflow完整代码
生活随笔
收集整理的這篇文章主要介紹了
【神经网络】(16) MobileNetV3 代码复现，网络解析，附Tensorflow完整代码
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
各位同學好，今天和大家分享一下如何使用 Tensorflow 構建 MobileNetV3 輕量化網絡模型。
MobileNetV3 做了如下改動（1）更新了V2中的逆轉殘差結構；（2）使用NAS搜索參數的技術；（3）重新設計耗時層結構。MobileNetV3相比V2版本，在圖像分類任務上，準確率上升了3.2%，延誤降低了20%
MobileNetV3 是對MobileNetV2的改進，建議大家在學習之前，先了解 MobileNetV1、V2。
MobileNetV1：https://blog.csdn.net/dgvv4/article/details/123415708
MobileNetV2：https://blog.csdn.net/dgvv4/article/details/123417739
?1. 網絡核心模塊介紹
1.1 MobileNetV1 深度可分離卷積
MobileNetV1 中主要使用了深度可分離卷積模塊，大大減少了參數量和計算量。
普通卷積是一個卷積核處理所有的通道，輸入特征圖有多少個通道，卷積核就有幾個通道，一個卷積核生成一張特征圖。
深度可分離卷積 可理解為 深度卷積 + 逐點卷積
深度卷積只處理長寬方向的空間信息；逐點卷積只處理跨通道方向的信息。能大大減少參數量，提高計算效率
深度卷積： 一個卷積核只處理一個通道，即每個卷積核只處理自己對應的通道。輸入特征圖有多少個通道就有多少個卷積核。將每個卷積核處理后的特征圖堆疊在一起。輸入和輸出特征圖的通道數相同。
由于只處理長寬方向的信息會導致丟失跨通道信息，為了將跨通道的信息補充回來，需要進行逐點卷積。
逐點卷積： 是使用1x1卷積對跨通道維度處理，有多少個1x1卷積核就會生成多少個特征圖。
1.2 MobileNetV2 逆轉殘差結構
MobileNetV2 使用了逆轉殘差模塊。輸入圖像，先使用1x1卷積提升通道數；然后在高維空間下使用深度卷積；再使用1x1卷積下降通道數，降維時采用線性激活函數(y=x)。當步長等于1且輸入和輸出特征圖的shape相同時，使用殘差連接輸入和輸出；當步長=2（下采樣階段）直接輸出降維后的特征圖。
對比 ResNet 的殘差結構。輸入圖像，先使用1x1卷積下降通道數；然后在低維空間下使用標準卷積，再使用1x1卷積上升通道數，激活函數都是ReLU函數。當步長等于1且輸入和輸出特征圖的shape相同時，使用殘差連接輸入和輸出；當步長=2（下采樣階段）直接輸出降維后的特征圖。
1.3 MobileNetV3 改進逆轉殘差結構
主要有以下改進:（1）添加SE注意力機制；（2）使用新的激活函數
1.3.1 SE注意力機制
（1）先將特征圖進行全局平均池化，特征圖有多少個通道，那么池化結果（一維向量）就有多少個元素，[h, w, c]==>[None, c]。
（2）然后經過兩個全連接層得到輸出向量。第一個全連接層的輸出通道數等于原輸入特征圖的通道數的1/4；第二個全連接層的輸出通道數等于原輸入特征圖的通道數。即先降維后升維。
（3）全連接層的輸出向量可理解為，向量的每個元素是對每張特征圖進行分析得出的權重關系。比較重要的特征圖就會賦予更大的權重，即該特征圖對應的向量元素的值較大。反之，不太重要的特征圖對應的權重值較小。
（4）第一個全連接層使用ReLU激活函數，第二個全連接層使用 hard_sigmoid 激活函數
（5）經過兩個全連接層得到一個由channel個元素組成的向量，每個元素是針對每個通道的權重，將權重和原特征圖的對應相乘，得到新的特征圖數據
以下圖為例，特征圖經過兩個全連接層之后，比較重要的特征圖對應的向量元素的值就較大。將得到的權重和對應特征圖中的所有元素相乘，得到新的輸出特征圖
1.3.2 使用不同的激活函數
swish激活函數公式為：，盡管提高了網絡精度，但是它的計算、求導復雜，對量化過程不友好，尤其對移動端設備的計算。
h_sigmoid激活函數公式為：，ReLU6激活函數公式為：
h_swish激活函數公式為：，替換之后網絡的推理速度加快，對量化過程比較友好
1.3.3 總體流程
圖像輸入，先通過1x1卷積上升通道數；然后在高維空間下使用深度卷積；再經過SE注意力機制優化特征圖數據；最后經過1x1卷積下降通道數（使用線性激活函數）。當步長等于1且輸入和輸出特征圖的shape相同時，使用殘差連接輸入和輸出；當步長=2（下采樣階段）直接輸出降維后的特征圖。
1.4 重新設計耗時層結構
（1）減少第一個卷積層的卷積核個數。將卷積核個數從32個降低到16個之后，準確率和降低之前是一樣的。減少卷積核個數可以減少計算量，節省2ms時間
（2）簡化最后的輸出層。刪除多余的卷積層，在準確率上沒有變化，節省了7ms執行時間，這7ms占據了整個推理過程的11%的執行時間。明顯提升計算速度。
2. 代碼復現
2.1 網絡結構圖
網絡模型結構如圖所示。exp size 代表1*1卷積上升的通道數；#out 代表1*1卷積下降的通道數，即輸出特征圖數量；SE 代表是否使用注意力機制；NL 代表使用哪種激活函數；s 代表步長；bneck 代表逆殘差結構；NBN 代表不使用批標準化。
2.2 搭建核心模塊
（1）激活函數選擇
根據上面的公式，定義hard_sigmoid激活函數和hard_swish激活函數。
#（1）激活函數：h-sigmoid
def h_sigmoid(input_tensor):x = layers.Activation('hard_sigmoid')(input_tensor)return x#（2）激活函數：h-swish
def h_swish(input_tensor):x = input_tensor * h_sigmoid(input_tensor)return x（2）SE注意力機制
SE注意力機制由 全局平均池化 + 全連接層降維 + 全連接層升維 + 對應權重相乘 組成。為了減少參數量和計算量，全連接層由1*1普通卷積層代替
#（3）SE注意力機制
def se_block(input_tensor):squeeze = input_tensor.shape[-1]/4 # 第一個全連接通道數下降1/4excitation = input_tensor.shape[-1] # 第二個全連接通道數上升至原來# 全局平均池化[b,h,w,c]==>[b,c]x = layers.GlobalAveragePooling2D()(input_tensor)# 添加寬度和高度的維度信息，因為下面要使用卷積層代替全連接層x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x) #[b,c]==>[b,1,1,c]# 第一個全連接層下降通道數，用 1*1卷積層 代替，減少參數量x = layers.Conv2D(filters=squeeze, # 通道數下降原來的1/4kernel_size=(1,1), # 1*1卷積融合通道信息strides=1, # 步長=1padding='same')(x) # 卷積過程中特征圖size不變x = layers.ReLU()(x) # relu激活# 第二個全連接層上升通道數，也用1*1卷積層代替x = layers.Conv2D(filters=excitation, # 通道數上升至原始的特征圖數量kernel_size=(1,1),strides=1,padding='same')(x)x = h_sigmoid(x) # hard_sigmoid激活函數# 將輸入特征圖的每個通道和SE得到的針對每個通道的權重相乘output = layers.Multiply()([input_tensor, x])return output（3）標準卷積塊
一個標準卷積塊是由 普通卷積 + 批標準化 + 激活函數 組成
#（4）標準卷積塊
def conv_block(input_tensor, filters, kernel_size, stride, activation):# 判斷使用什么類型的激活函數if activation == 'RE':act = layers.ReLU() # relu激活elif activation == 'HS':act = h_swish # hardswish激活# 普通卷積x = layers.Conv2D(filters, kernel_size, strides=stride, padding='same', use_bias=False)(input_tensor)# BN層x = layers.BatchNormalization()(x)# 激活x = act(x)return x（4）逆殘差模塊
相比于MobileNetV2的逆殘差模塊，添加了注意力機制，使用不同的激活函數
#（5）逆轉殘差模塊bneck
def bneck(x, expansion, filters, kernel_size, stride, se, activation):"""filters代表bottleneck模塊輸出特征圖的通道數個數se是bool類型, se=True 就使用注意力機制, 反之則不使用activation表示使用什么類型的激活函數'RE'和'HS'"""# 殘差邊residual = x# 判斷使用什么類型的激活函數if activation == 'RE':act = layers.ReLU() # relu激活elif activation == 'HS':act = h_swish # hardswish激活# ① 1*1卷積上升通道數if expansion != filters: # 第一個bneck模塊不需要上升通道數，x = layers.Conv2D(filters = expansion, # 上升的通道數kernel_size = (1,1), # 1*1卷積融合通道信息strides = 1, # 只處理通道方向的信息padding = 'same', # 卷積過程中size不變，use_bias = False)(x) # 有BN層就不使用偏置x = layers.BatchNormalization()(x) # 批標準化x = act(x) # 激活函數# ② 深度卷積x = layers.DepthwiseConv2D(kernel_size = kernel_size, # 卷積核sizestrides = stride, # 是否進行下采樣padding = 'same', # 步長=2，特征圖長寬減半use_bias = False)(x) # 有BN層就不要偏置x = layers.BatchNormalization()(x) # 批標準化x = act(x) # 激活函數# ③ 是否使用注意力機制if se == True:x = se_block(x)# ④ 1*1卷積下降通道數x = layers.Conv2D(filters = filters, # 輸出特征圖個數kernel_size = (1,1), # 1*1卷積融合通道信息strides = 1,padding = 'same',use_bias = False)(x)x = layers.BatchNormalization()(x)# 使用的是線性激活函數y=x# ④ 如果深度卷積的步長=1并且輸入和輸出的shape相同，就疊加殘差邊if stride == 1 and residual.shape==x.shape:x = layers.Add()([residual, x])return x # 如果步長=2，直接返回1*1卷積下降通道數后的結果2.3 完整代碼展示
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Model, layers#（1）激活函數：h-sigmoid
def h_sigmoid(input_tensor):x = layers.Activation('hard_sigmoid')(input_tensor)return x#（2）激活函數：h-swish
def h_swish(input_tensor):x = input_tensor * h_sigmoid(input_tensor)return x#（3）SE注意力機制
def se_block(input_tensor):squeeze = input_tensor.shape[-1]/4 # 第一個全連接通道數下降1/4excitation = input_tensor.shape[-1] # 第二個全連接通道數上升至原來# 全局平均池化[b,h,w,c]==>[b,c]x = layers.GlobalAveragePooling2D()(input_tensor)# 添加寬度和高度的維度信息，因為下面要使用卷積層代替全連接層x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x) #[b,c]==>[b,1,1,c]# 第一個全連接層下降通道數，用 1*1卷積層 代替，減少參數量x = layers.Conv2D(filters=squeeze, # 通道數下降原來的1/4kernel_size=(1,1), # 1*1卷積融合通道信息strides=1, # 步長=1padding='same')(x) # 卷積過程中特征圖size不變x = layers.ReLU()(x) # relu激活# 第二個全連接層上升通道數，也用1*1卷積層代替x = layers.Conv2D(filters=excitation, # 通道數上升至原始的特征圖數量kernel_size=(1,1),strides=1,padding='same')(x)x = h_sigmoid(x) # hard_sigmoid激活函數# 將輸入特征圖的每個通道和SE得到的針對每個通道的權重相乘output = layers.Multiply()([input_tensor, x])return output#（4）標準卷積塊
def conv_block(input_tensor, filters, kernel_size, stride, activation):# 判斷使用什么類型的激活函數if activation == 'RE':act = layers.ReLU() # relu激活elif activation == 'HS':act = h_swish # hardswish激活# 普通卷積x = layers.Conv2D(filters, kernel_size, strides=stride, padding='same', use_bias=False)(input_tensor)# BN層x = layers.BatchNormalization()(x)# 激活x = act(x)return x#（5）逆轉殘差模塊bneck
def bneck(x, expansion, filters, kernel_size, stride, se, activation):"""filters代表bottleneck模塊輸出特征圖的通道數個數se是bool類型, se=True 就使用注意力機制, 反之則不使用activation表示使用什么類型的激活函數'RE'和'HS'"""# 殘差邊residual = x# 判斷使用什么類型的激活函數if activation == 'RE':act = layers.ReLU() # relu激活elif activation == 'HS':act = h_swish # hardswish激活# ① 1*1卷積上升通道數if expansion != filters: # 第一個bneck模塊不需要上升通道數，x = layers.Conv2D(filters = expansion, # 上升的通道數kernel_size = (1,1), # 1*1卷積融合通道信息strides = 1, # 只處理通道方向的信息padding = 'same', # 卷積過程中size不變，use_bias = False)(x) # 有BN層就不使用偏置x = layers.BatchNormalization()(x) # 批標準化x = act(x) # 激活函數# ② 深度卷積x = layers.DepthwiseConv2D(kernel_size = kernel_size, # 卷積核sizestrides = stride, # 是否進行下采樣padding = 'same', # 步長=2，特征圖長寬減半use_bias = False)(x) # 有BN層就不要偏置x = layers.BatchNormalization()(x) # 批標準化x = act(x) # 激活函數# ③ 是否使用注意力機制if se == True:x = se_block(x)# ④ 1*1卷積下降通道數x = layers.Conv2D(filters = filters, # 輸出特征圖個數kernel_size = (1,1), # 1*1卷積融合通道信息strides = 1,padding = 'same',use_bias = False)(x)x = layers.BatchNormalization()(x)# 使用的是線性激活函數y=x# ④ 如果深度卷積的步長=1并且輸入和輸出的shape相同，就疊加殘差邊if stride == 1 and residual.shape==x.shape:x = layers.Add()([residual, x])return x # 如果步長=2，直接返回1*1卷積下降通道數后的結果#（6）主干網絡
def mobilenet(input_shape, classes): # 輸入圖像shape，分類數# 構造輸入inputs = keras.Input(shape=input_shape)# [224,224,3] ==> [112,112,16]x = conv_block(inputs, filters=16, kernel_size=(3,3), stride=2, activation='HS')# [112,112,16] ==> [112,112,16]x = bneck(x, expansion=16, filters=16, kernel_size=(3,3), stride=1, se=False, activation='RE')# [112,112,16] ==> [56,56,24]x = bneck(x, expansion=64, filters=24, kernel_size=(3,3), stride=2, se=False, activation='RE')# [56,56,24] ==> [56,56,24]x = bneck(x, expansion=72, filters=24, kernel_size=(3,3), stride=1, se=False, activation='RE')# [56,56,24] ==> [28,28,40]x = bneck(x, expansion=72, filters=40, kernel_size=(5,5), stride=2, se=True, activation='RE')# [28,28,40] ==> [28,28,40]x = bneck(x, expansion=120, filters=40, kernel_size=(5,5), stride=1, se=True, activation='RE')# [28,28,40] ==> [28,28,40]x = bneck(x, expansion=120, filters=40, kernel_size=(5,5), stride=1, se=True, activation='RE')# [28,28,40] ==> [14,14,80]x = bneck(x, expansion=240, filters=80, kernel_size=(3,3), stride=2, se=False, activation='HS')# [14,14,80] ==> [14,14,80]x = bneck(x, expansion=200, filters=80, kernel_size=(3,3), stride=1, se=False, activation='HS')# [14,14,80] ==> [14,14,80]x = bneck(x, expansion=184, filters=80, kernel_size=(3,3), stride=1, se=False, activation='HS')# [14,14,80] ==> [14,14,80]x = bneck(x, expansion=184, filters=80, kernel_size=(3,3), stride=1, se=False, activation='HS')# [14,14,80] ==> [14,14,112]x = bneck(x, expansion=480, filters=112, kernel_size=(3,3), stride=1, se=True, activation='HS')# [14,14,112] ==> [14,14,112]x = bneck(x, expansion=672, filters=112, kernel_size=(3,3), stride=1, se=True, activation='HS')# [14,14,112] ==> [7,7,160]x = bneck(x, expansion=672, filters=160, kernel_size=(5,5), stride=2, se=True, activation='HS')# [7,7,160] ==> [7,7,160]x = bneck(x, expansion=960, filters=160, kernel_size=(5,5), stride=1, se=True, activation='HS')# [7,7,160] ==> [7,7,160]x = bneck(x, expansion=960, filters=160, kernel_size=(5,5), stride=1, se=True, activation='HS')# [7,7,160] ==> [7,7,960]x = conv_block(x, filters=960, kernel_size=(1,1), stride=1, activation='HS')# [7,7,960] ==> [None,960]x = layers.MaxPooling2D(pool_size=(7,7))(x)# [None,960] ==> [1,1,960]x = layers.Reshape(target_shape=(1,1,x.shape[-1]))(x)# [1,1,960] ==> [1,1,1280]x = layers.Conv2D(filters=1280, kernel_size=(1,1), strides=1, padding='same')(x)x = h_swish(x)# [1,1,960] ==> [1,1,classes]x = layers.Conv2D(filters=classes, kernel_size=(1,1), strides=1, padding='same')(x)# [1,1,classes] ==> [None,classes]logits = layers.Flatten()(x)# 構造模型model = Model(inputs, logits)return model#（7）接收網絡模型
if __name__ == '__main__':model = mobilenet(input_shape=[224,224,3], classes=1000)model.summary() # 查看網絡架構2.4 查看網絡模型結構
通過函數model.summary()查看網絡總體框架，約五百萬參數量
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 112, 112, 16) 432 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 112, 112, 16) 64 conv2d[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, 112, 112, 16) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
tf.math.multiply (TFOpLambda) (None, 112, 112, 16) 0 batch_normalization[0][0] activation[0][0]
__________________________________________________________________________________________________
depthwise_conv2d (DepthwiseConv (None, 112, 112, 16) 144 tf.math.multiply[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 112, 112, 16) 64 depthwise_conv2d[0][0]
__________________________________________________________________________________________________
re_lu (ReLU) (None, 112, 112, 16) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 112, 112, 16) 256 re_lu[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 112, 112, 16) 64 conv2d_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 112, 112, 64) 1024 batch_normalization_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 112, 112, 64) 256 conv2d_2[0][0]
__________________________________________________________________________________________________
re_lu_1 (ReLU) multiple 0 batch_normalization_3[0][0] batch_normalization_4[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_1 (DepthwiseCo (None, 56, 56, 64) 576 re_lu_1[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 56, 56, 64) 256 depthwise_conv2d_1[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 56, 56, 24) 1536 re_lu_1[1][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 56, 56, 24) 96 conv2d_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 56, 56, 72) 1728 batch_normalization_5[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 56, 56, 72) 288 conv2d_4[0][0]
__________________________________________________________________________________________________
re_lu_2 (ReLU) (None, 56, 56, 72) 0 batch_normalization_6[0][0] batch_normalization_7[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_2 (DepthwiseCo (None, 56, 56, 72) 648 re_lu_2[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 56, 56, 72) 288 depthwise_conv2d_2[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 56, 56, 24) 1728 re_lu_2[1][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 56, 56, 24) 96 conv2d_5[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 56, 56, 72) 1728 batch_normalization_8[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 56, 56, 72) 288 conv2d_6[0][0]
__________________________________________________________________________________________________
re_lu_3 (ReLU) multiple 0 batch_normalization_9[0][0] batch_normalization_10[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 72) 1800 re_lu_3[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 28, 28, 72) 288 depthwise_conv2d_3[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 72) 0 re_lu_3[1][0]
__________________________________________________________________________________________________
reshape (Reshape) (None, 1, 1, 72) 0 global_average_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 1, 1, 18) 1314 reshape[0][0]
__________________________________________________________________________________________________
re_lu_4 (ReLU) (None, 1, 1, 18) 0 conv2d_7[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 1, 1, 72) 1368 re_lu_4[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 1, 1, 72) 0 conv2d_8[0][0]
__________________________________________________________________________________________________
multiply (Multiply) (None, 28, 28, 72) 0 re_lu_3[1][0] activation_1[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 28, 28, 40) 2880 multiply[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 28, 28, 40) 160 conv2d_9[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 28, 28, 120) 4800 batch_normalization_11[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 28, 28, 120) 480 conv2d_10[0][0]
__________________________________________________________________________________________________
re_lu_5 (ReLU) (None, 28, 28, 120) 0 batch_normalization_12[0][0] batch_normalization_13[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_4 (DepthwiseCo (None, 28, 28, 120) 3000 re_lu_5[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 28, 28, 120) 480 depthwise_conv2d_4[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_1 (Glo (None, 120) 0 re_lu_5[1][0]
__________________________________________________________________________________________________
reshape_1 (Reshape) (None, 1, 1, 120) 0 global_average_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 1, 1, 30) 3630 reshape_1[0][0]
__________________________________________________________________________________________________
re_lu_6 (ReLU) (None, 1, 1, 30) 0 conv2d_11[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 1, 1, 120) 3720 re_lu_6[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, 1, 1, 120) 0 conv2d_12[0][0]
__________________________________________________________________________________________________
multiply_1 (Multiply) (None, 28, 28, 120) 0 re_lu_5[1][0] activation_2[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 28, 28, 40) 4800 multiply_1[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 28, 28, 40) 160 conv2d_13[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 28, 28, 120) 4800 batch_normalization_14[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 28, 28, 120) 480 conv2d_14[0][0]
__________________________________________________________________________________________________
re_lu_7 (ReLU) (None, 28, 28, 120) 0 batch_normalization_15[0][0] batch_normalization_16[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_5 (DepthwiseCo (None, 28, 28, 120) 3000 re_lu_7[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 28, 28, 120) 480 depthwise_conv2d_5[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_2 (Glo (None, 120) 0 re_lu_7[1][0]
__________________________________________________________________________________________________
reshape_2 (Reshape) (None, 1, 1, 120) 0 global_average_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 1, 1, 30) 3630 reshape_2[0][0]
__________________________________________________________________________________________________
re_lu_8 (ReLU) (None, 1, 1, 30) 0 conv2d_15[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 1, 1, 120) 3720 re_lu_8[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, 1, 1, 120) 0 conv2d_16[0][0]
__________________________________________________________________________________________________
multiply_2 (Multiply) (None, 28, 28, 120) 0 re_lu_7[1][0] activation_3[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 28, 28, 40) 4800 multiply_2[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 28, 28, 40) 160 conv2d_17[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 28, 28, 240) 9600 batch_normalization_17[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 28, 28, 240) 960 conv2d_18[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 28, 28, 240) 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
tf.math.multiply_1 (TFOpLambda) (None, 28, 28, 240) 0 batch_normalization_18[0][0] activation_4[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 240) 2160 tf.math.multiply_1[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 14, 14, 240) 960 depthwise_conv2d_6[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 14, 14, 240) 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
tf.math.multiply_2 (TFOpLambda) (None, 14, 14, 240) 0 batch_normalization_19[0][0] activation_5[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 14, 14, 80) 19200 tf.math.multiply_2[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 14, 14, 80) 320 conv2d_19[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 14, 14, 200) 16000 batch_normalization_20[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 14, 14, 200) 800 conv2d_20[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, 14, 14, 200) 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
tf.math.multiply_3 (TFOpLambda) (None, 14, 14, 200) 0 batch_normalization_21[0][0] activation_6[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 200) 1800 tf.math.multiply_3[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 14, 14, 200) 800 depthwise_conv2d_7[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, 14, 14, 200) 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
tf.math.multiply_4 (TFOpLambda) (None, 14, 14, 200) 0 batch_normalization_22[0][0] activation_7[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 14, 14, 80) 16000 tf.math.multiply_4[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 14, 14, 80) 320 conv2d_21[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 14, 14, 184) 14720 batch_normalization_23[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 14, 14, 184) 736 conv2d_22[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, 14, 14, 184) 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
tf.math.multiply_5 (TFOpLambda) (None, 14, 14, 184) 0 batch_normalization_24[0][0] activation_8[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 184) 1656 tf.math.multiply_5[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 14, 14, 184) 736 depthwise_conv2d_8[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 14, 14, 184) 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
tf.math.multiply_6 (TFOpLambda) (None, 14, 14, 184) 0 batch_normalization_25[0][0] activation_9[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 14, 14, 80) 14720 tf.math.multiply_6[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 14, 14, 80) 320 conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 14, 14, 184) 14720 batch_normalization_26[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 14, 14, 184) 736 conv2d_24[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, 14, 14, 184) 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
tf.math.multiply_7 (TFOpLambda) (None, 14, 14, 184) 0 batch_normalization_27[0][0] activation_10[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 184) 1656 tf.math.multiply_7[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 184) 736 depthwise_conv2d_9[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, 14, 14, 184) 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
tf.math.multiply_8 (TFOpLambda) (None, 14, 14, 184) 0 batch_normalization_28[0][0] activation_11[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 14, 14, 80) 14720 tf.math.multiply_8[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 14, 14, 80) 320 conv2d_25[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 14, 14, 480) 38400 batch_normalization_29[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 14, 14, 480) 1920 conv2d_26[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, 14, 14, 480) 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
tf.math.multiply_9 (TFOpLambda) (None, 14, 14, 480) 0 batch_normalization_30[0][0] activation_12[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 480) 4320 tf.math.multiply_9[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 480) 1920 depthwise_conv2d_10[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, 14, 14, 480) 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
tf.math.multiply_10 (TFOpLambda (None, 14, 14, 480) 0 batch_normalization_31[0][0] activation_13[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_3 (Glo (None, 480) 0 tf.math.multiply_10[0][0]
__________________________________________________________________________________________________
reshape_3 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_3[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 1, 1, 120) 57720 reshape_3[0][0]
__________________________________________________________________________________________________
re_lu_9 (ReLU) (None, 1, 1, 120) 0 conv2d_27[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 1, 1, 480) 58080 re_lu_9[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, 1, 1, 480) 0 conv2d_28[0][0]
__________________________________________________________________________________________________
multiply_3 (Multiply) (None, 14, 14, 480) 0 tf.math.multiply_10[0][0] activation_14[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 14, 14, 112) 53760 multiply_3[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 112) 448 conv2d_29[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 14, 14, 672) 75264 batch_normalization_32[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 672) 2688 conv2d_30[0][0]
__________________________________________________________________________________________________
activation_15 (Activation) (None, 14, 14, 672) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
tf.math.multiply_11 (TFOpLambda (None, 14, 14, 672) 0 batch_normalization_33[0][0] activation_15[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 672) 6048 tf.math.multiply_11[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 672) 2688 depthwise_conv2d_11[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, 14, 14, 672) 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
tf.math.multiply_12 (TFOpLambda (None, 14, 14, 672) 0 batch_normalization_34[0][0] activation_16[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_4 (Glo (None, 672) 0 tf.math.multiply_12[0][0]
__________________________________________________________________________________________________
reshape_4 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_4[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 1, 1, 168) 113064 reshape_4[0][0]
__________________________________________________________________________________________________
re_lu_10 (ReLU) (None, 1, 1, 168) 0 conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 1, 1, 672) 113568 re_lu_10[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, 1, 1, 672) 0 conv2d_32[0][0]
__________________________________________________________________________________________________
multiply_4 (Multiply) (None, 14, 14, 672) 0 tf.math.multiply_12[0][0] activation_17[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 14, 14, 112) 75264 multiply_4[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 112) 448 conv2d_33[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 14, 14, 672) 75264 batch_normalization_35[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 672) 2688 conv2d_34[0][0]
__________________________________________________________________________________________________
activation_18 (Activation) (None, 14, 14, 672) 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
tf.math.multiply_13 (TFOpLambda (None, 14, 14, 672) 0 batch_normalization_36[0][0] activation_18[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_12 (DepthwiseC (None, 7, 7, 672) 16800 tf.math.multiply_13[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 7, 7, 672) 2688 depthwise_conv2d_12[0][0]
__________________________________________________________________________________________________
activation_19 (Activation) (None, 7, 7, 672) 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
tf.math.multiply_14 (TFOpLambda (None, 7, 7, 672) 0 batch_normalization_37[0][0] activation_19[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_5 (Glo (None, 672) 0 tf.math.multiply_14[0][0]
__________________________________________________________________________________________________
reshape_5 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_5[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 1, 1, 168) 113064 reshape_5[0][0]
__________________________________________________________________________________________________
re_lu_11 (ReLU) (None, 1, 1, 168) 0 conv2d_35[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 1, 1, 672) 113568 re_lu_11[0][0]
__________________________________________________________________________________________________
activation_20 (Activation) (None, 1, 1, 672) 0 conv2d_36[0][0]
__________________________________________________________________________________________________
multiply_5 (Multiply) (None, 7, 7, 672) 0 tf.math.multiply_14[0][0] activation_20[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, 7, 7, 160) 107520 multiply_5[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 7, 7, 160) 640 conv2d_37[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, 7, 7, 960) 153600 batch_normalization_38[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 7, 7, 960) 3840 conv2d_38[0][0]
__________________________________________________________________________________________________
activation_21 (Activation) (None, 7, 7, 960) 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
tf.math.multiply_15 (TFOpLambda (None, 7, 7, 960) 0 batch_normalization_39[0][0] activation_21[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_13 (DepthwiseC (None, 7, 7, 960) 24000 tf.math.multiply_15[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 7, 7, 960) 3840 depthwise_conv2d_13[0][0]
__________________________________________________________________________________________________
activation_22 (Activation) (None, 7, 7, 960) 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
tf.math.multiply_16 (TFOpLambda (None, 7, 7, 960) 0 batch_normalization_40[0][0] activation_22[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_6 (Glo (None, 960) 0 tf.math.multiply_16[0][0]
__________________________________________________________________________________________________
reshape_6 (Reshape) (None, 1, 1, 960) 0 global_average_pooling2d_6[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 1, 1, 240) 230640 reshape_6[0][0]
__________________________________________________________________________________________________
re_lu_12 (ReLU) (None, 1, 1, 240) 0 conv2d_39[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 1, 1, 960) 231360 re_lu_12[0][0]
__________________________________________________________________________________________________
activation_23 (Activation) (None, 1, 1, 960) 0 conv2d_40[0][0]
__________________________________________________________________________________________________
multiply_6 (Multiply) (None, 7, 7, 960) 0 tf.math.multiply_16[0][0] activation_23[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 7, 7, 160) 153600 multiply_6[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 7, 7, 160) 640 conv2d_41[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 7, 7, 960) 153600 batch_normalization_41[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 7, 7, 960) 3840 conv2d_42[0][0]
__________________________________________________________________________________________________
activation_24 (Activation) (None, 7, 7, 960) 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
tf.math.multiply_17 (TFOpLambda (None, 7, 7, 960) 0 batch_normalization_42[0][0] activation_24[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 960) 24000 tf.math.multiply_17[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 960) 3840 depthwise_conv2d_14[0][0]
__________________________________________________________________________________________________
activation_25 (Activation) (None, 7, 7, 960) 0 batch_normalization_43[0][0]
__________________________________________________________________________________________________
tf.math.multiply_18 (TFOpLambda (None, 7, 7, 960) 0 batch_normalization_43[0][0] activation_25[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_7 (Glo (None, 960) 0 tf.math.multiply_18[0][0]
__________________________________________________________________________________________________
reshape_7 (Reshape) (None, 1, 1, 960) 0 global_average_pooling2d_7[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 1, 1, 240) 230640 reshape_7[0][0]
__________________________________________________________________________________________________
re_lu_13 (ReLU) (None, 1, 1, 240) 0 conv2d_43[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 1, 1, 960) 231360 re_lu_13[0][0]
__________________________________________________________________________________________________
activation_26 (Activation) (None, 1, 1, 960) 0 conv2d_44[0][0]
__________________________________________________________________________________________________
multiply_7 (Multiply) (None, 7, 7, 960) 0 tf.math.multiply_18[0][0] activation_26[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, 7, 7, 160) 153600 multiply_7[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 160) 640 conv2d_45[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, 7, 7, 960) 153600 batch_normalization_44[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 960) 3840 conv2d_46[0][0]
__________________________________________________________________________________________________
activation_27 (Activation) (None, 7, 7, 960) 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
tf.math.multiply_19 (TFOpLambda (None, 7, 7, 960) 0 batch_normalization_45[0][0] activation_27[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 1, 1, 960) 0 tf.math.multiply_19[0][0]
__________________________________________________________________________________________________
reshape_8 (Reshape) (None, 1, 1, 960) 0 max_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, 1, 1, 1280) 1230080 reshape_8[0][0]
__________________________________________________________________________________________________
activation_28 (Activation) (None, 1, 1, 1280) 0 conv2d_47[0][0]
__________________________________________________________________________________________________
tf.math.multiply_20 (TFOpLambda (None, 1, 1, 1280) 0 conv2d_47[0][0] activation_28[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, 1, 1, 1000) 1281000 tf.math.multiply_20[0][0]
__________________________________________________________________________________________________
flatten (Flatten) (None, 1000) 0 conv2d_48[0][0]
==================================================================================================
Total params: 5,505,598
Trainable params: 5,481,198
Non-trainable params: 24,400
__________________________________________________________________________________________________總結
以上是生活随笔為你收集整理的【神经网络】(16) MobileNetV3 代码复现，网络解析，附Tensorflow完整代码的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 【深度学习理论】(1) 损失函数
- 下一篇： 【机器视觉案例】(13) 脸部和摄像机间