引言

本文主要用于對7個經(jīng)典卷積神經(jīng)網(wǎng)絡的初識，大致了解各個網(wǎng)絡提出的背景，以及各自對卷積神經(jīng)網(wǎng)絡發(fā)展的作用（即網(wǎng)絡的特點）。

• 經(jīng)典的卷積神經(jīng)網(wǎng)絡：
  • LeNet
  • AlexNet
  • ZF Net
  • VGG
  • GoogLeNet
  • ResNet
  • DenseNet

參考： CNN網(wǎng)絡架構演進：從LeNet到DenseNet（本文內(nèi)容來源，強力推薦）

LeNet（5層）

• 提出背景：LeCun在1998年提出，用于解決手寫數(shù)字識別的視覺任務。

• 歷史意義：定義了CNN的基本組件，是CNN的鼻祖。自此，CNN的最基本的架構確定下來：卷積層、池化層、全連接層。如今各大深度學習框架中所使用的LeNet都是簡化改進過的LeNet-5（-5表示具有5個層），和原始的LeNet有些許不同，比如把激活函數(shù)改為了現(xiàn)在很常用的ReLu。

• 經(jīng)典的LeNet-5網(wǎng)絡結構：如下圖。

• 數(shù)據(jù)矩陣計算：
  • 輸入是單通道的28*28大小矩陣，用矩陣表示就是[1,28,28]；
  • cov1：20個5*5的卷積核，滑動步長為1，經(jīng)過該層后尺寸變?yōu)?4，28-5+1=24，輸出矩陣為[20,24,24]；
  • pool1：2*2，步長2，池化操作后，尺寸減半，變?yōu)?2×12，輸出矩陣為[20,12,12]；
  • cov2：50個5*5的卷積核，步長1，卷積后尺寸變?yōu)?,12-5+1=8，輸出矩陣為[50,8,8]；
  • pool2：2*2，步長2，池化操作后，尺寸減半，變?yōu)?×4，輸出矩陣為[50,4,4]；
  • fc1：輸入為[50,4,4]，神經(jīng)元數(shù)目為500，再接relu激活函數(shù)，輸出為500；
  • fc2：輸入為500，神經(jīng)元個數(shù)為10，得到10維的特征向量；
  • output：輸入為10維的特征向量，送入softmaxt分類，得到分類結果的概率output。

    cov為卷積層，pool為池化層，fc的全連接層。

AlexNet（8層）

• 提出背景：AlexNet在2012年ImageNet競賽中以超過第二名10.9個百分點的絕對優(yōu)勢一舉奪冠，從此深度學習和卷積神經(jīng)網(wǎng)絡名聲鵲起，深度學習的研究如雨后春筍般出現(xiàn)，AlexNet的出現(xiàn)可謂是卷積神經(jīng)網(wǎng)絡的王者歸來。

• 歷史意義：

  • 更深的網(wǎng)絡
  • 數(shù)據(jù)增廣：數(shù)據(jù)增廣技巧來增加模型泛化能力；
  • ReLU：ReLU代替Sigmoid來加快SGD的收斂速度；
  • dropout：有效緩解了模型的過擬合；
  • LRN：局部響應歸一化，減小高激活神經(jīng)元的使用，后發(fā)現(xiàn)這個不起作用，目前已棄用。

Dropout原理類似于淺層學習算法的中集成算法，該方法通過讓全連接層的神經(jīng)元（該模型在前兩個全連接層引入Dropout）以一定的概率失去活性（比如0.5）失活的神經(jīng)元不再參與前向和反向傳播，相當于約有一半的神經(jīng)元不再起作用。在測試的時候，讓所有神經(jīng)元的輸出乘0.5。Dropout的引用，有效緩解了模型的過擬合。

Local Responce Normalization（LRN）：局部響應歸一層的基本思路是，假如這是網(wǎng)絡的一塊，比如是 13×13×256， LRN 要做的就是選取一個位置，比如說這樣一個位置，從這個位置穿過整個通道，能得到 256 個數(shù)字，并進行歸一化。進行局部響應歸一化的動機是，對于這張 13×13 的圖像中的每個位置來說，我們可能并不需要太多的高激活神經(jīng)元。但是后來，很多研究者發(fā)現(xiàn) LRN 起不到太大作用，因為并不重要，而且我們現(xiàn)在并不用 LRN 來訓練網(wǎng)絡。

ZF-Net（8層）

• 提出背景：ZFNet是2013ImageNet分類任務的冠軍，其網(wǎng)絡結構沒什么改進，主要是參數(shù)的變化，性能較Alex提升了不少。ZF-Net只是將AlexNet第一層卷積核由11變成7，步長由4變?yōu)?，第3，4，5卷積層轉變?yōu)?84，384，256。

VGG-Nets（19層）

• 提出背景：VGG-Nets是由牛津大學VGG（Visual Geometry Group）提出，是2014年ImageNet競賽定位任務的第一名和分類任務的第二名的中的基礎網(wǎng)絡。VGG可以看成是加深版本的AlexNet. 都是conv layer + FC layer，在當時看來這是一個非常深的網(wǎng)絡了，因為層數(shù)高達十多層，我們從其論文名字就知道了（《Very Deep Convolutional Networks for Large-Scale Visual Recognition》），當然以現(xiàn)在的目光看來VGG真的稱不上是一個very deep的網(wǎng)絡。

• 歷史意義：

  • 卷積層使用更小的filter尺寸和間隔：VGG-Nets，用到的卷積核的尺寸無非都是1×1和3×3的小卷積核，可以替代大的filter尺寸；

3×3卷積核的優(yōu)點：

• 多個3×3的卷基層比一個大尺寸filter卷基層有更多的非線性，使得判決函數(shù)更加具有判決性
• 多個3×3的卷積層比一個大尺寸的filter有更少的參數(shù)，假設卷基層的輸入和輸出的特征圖大小相同為C，那么三個3×3的卷積層參數(shù)個數(shù)3×（3×3×C×C）=27CC；一個7×7的卷積層參數(shù)為49CC；所以可以把三個3×3的filter看成是一個7×7filter的分解（中間層有非線性的分解）

1*1卷積核的優(yōu)點：作用是在不影響輸入輸出維數(shù)的情況下，對輸入進行線性形變，然后通過Relu進行非線性處理，增加網(wǎng)絡的非線性表達能力。

GoogLeNet（22層）

• 提出背景：GoogLeNet在2014的ImageNet分類任務上擊敗了VGG-Nets奪得冠軍，其實力肯定是非常深厚的，GoogLeNet跟AlexNet,VGG-Nets這種單純依靠加深網(wǎng)絡結構進而改進網(wǎng)絡性能的思路不一樣，它另辟幽徑，在加深網(wǎng)絡的同時（22層），也在網(wǎng)絡結構上做了創(chuàng)新，引入Inception結構代替了單純的卷積+激活的傳統(tǒng)操作（這思路最早由Network in Network提出）。GoogLeNet進一步把對卷積神經(jīng)網(wǎng)絡的研究推上新的高度。

• 歷史意義：
  • 引入Inception結構；
  • 中間層的輔助LOSS單元；
  • 后面的全連接層全部替換為簡單的全局平均pooling。

ResNet（152層）

• 提出背景：2015年何愷明推出的ResNet在ISLVRC和COCO上橫掃所有選手，獲得冠軍。ResNet在網(wǎng)絡結構上做了大創(chuàng)新，而不再是簡單的堆積層數(shù)，ResNet在卷積神經(jīng)網(wǎng)絡的新思路，絕對是深度學習發(fā)展歷程上里程碑式的事件。

• 歷史意義：

  • 層數(shù)非常深，已經(jīng)超過百層；
  • 引入殘差單元來解決退化問題；

• 維度匹配方案：

  • zero_padding:對恒等層進行0填充的方式將維度補充完整，這種方法不會增加額外的參數(shù)；
  • projection:在恒等層采用1x1的卷積核來增加維度。這種方法會增加額外的參數(shù)。

DenseNet

• 提出背景：自Resnet提出以后，ResNet的變種網(wǎng)絡層出不窮，都各有其特點，網(wǎng)絡性能也有一定的提升。本文介紹的最后一個網(wǎng)絡是CVPR 2017最佳論文DenseNet，論文中提出的DenseNet（Dense Convolutional Network）主要還是和ResNet及Inception網(wǎng)絡做對比，思想上有借鑒，但卻是全新的結構，網(wǎng)絡結構并不復雜，卻非常有效，在CIFAR指標上全面超越ResNet?？梢哉fDenseNet吸收了ResNet最精華的部分，并在此上做了更加創(chuàng)新的工作，使得網(wǎng)絡性能進一步提升。

• 歷史意義：密集連接：緩解梯度消失問題，加強特征傳播，鼓勵特征復用，極大的減少了參數(shù)量

在同層深度下獲得更好的收斂率，DenseNet具有非常強大的內(nèi)存占用。

keras代碼實現(xiàn)：

• LeNet的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10

  def LeNet(): model = Sequential() model.add(Conv2D(32,(5,5),strides=(1,1),input_shape=(28,28,1),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(64,(5,5),strides=(1,1),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Flatten()) model.add(Dense(100,activation='relu')) model.add(Dense(10,activation='softmax')) return model

• AlexNet的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

  def AlexNet(): model = Sequential() model.add(Conv2D(96,(11,11),strides=(4,4),input_shape=(227,227,3),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(256,(5,5),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Flatten()) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000,activation='softmax')) return model

• ZF-Net的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

  def ZF_Net(): model = Sequential() model.add(Conv2D(96,(7,7),strides=(2,2),input_shape=(224,224,3),padding='valid',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(256,(5,5),strides=(2,2),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(384,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(3,3),strides=(2,2))) model.add(Flatten()) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000,activation='softmax')) return model

• VGG-16的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

  def VGG_16(): model = Sequential() model.add(Conv2D(64,(3,3),strides=(1,1),input_shape=(224,224,3),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(64,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(128,(3,2),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(128,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(256,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(Conv2D(512,(3,3),strides=(1,1),padding='same',activation='relu',kernel_initializer='uniform')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Flatten()) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096,activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000,activation='softmax')) return model

• GoogLeNet的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

  def Conv2d_BN(x, nb_filter,kernel_size, padding='same',strides=(1,1),name=None): if name is not None: bn_name = name + '_bn' conv_name = name + '_conv' else: bn_name = None conv_name = None x = Conv2D(nb_filter,kernel_size,padding=padding,strides=strides,activation='relu',name=conv_name)(x) x = BatchNormalization(axis=3,name=bn_name)(x) return x def Inception(x,nb_filter): branch1x1 = Conv2d_BN(x,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branch3x3 = Conv2d_BN(x,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branch3x3 = Conv2d_BN(branch3x3,nb_filter,(3,3), padding='same',strides=(1,1),name=None) branch5x5 = Conv2d_BN(x,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branch5x5 = Conv2d_BN(branch5x5,nb_filter,(1,1), padding='same',strides=(1,1),name=None) branchpool = MaxPooling2D(pool_size=(3,3),strides=(1,1),padding='same')(x) branchpool = Conv2d_BN(branchpool,nb_filter,(1,1),padding='same',strides=(1,1),name=None) x = concatenate([branch1x1,branch3x3,branch5x5,branchpool],axis=3) return x def GoogLeNet(): inpt = Input(shape=(224,224,3)) #padding = 'same'，填充為(步長-1）/2,還可以用ZeroPadding2D((3,3)) x = Conv2d_BN(inpt,64,(7,7),strides=(2,2),padding='same') x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Conv2d_BN(x,192,(3,3),strides=(1,1),padding='same') x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Inception(x,64)#256 x = Inception(x,120)#480 x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Inception(x,128)#512 x = Inception(x,128) x = Inception(x,128) x = Inception(x,132)#528 x = Inception(x,208)#832 x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Inception(x,208) x = Inception(x,256)#1024 x = AveragePooling2D(pool_size=(7,7),strides=(7,7),padding='same')(x) x = Dropout(0.4)(x) x = Dense(1000,activation='relu')(x) x = Dense(1000,activation='softmax')(x) model = Model(inpt,x,name='inception') return model

• ResNet-50的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

  def Conv2d_BN(x, nb_filter,kernel_size, strides=(1,1), padding='same',name=None): if name is not None: bn_name = name + '_bn' conv_name = name + '_conv' else: bn_name = None conv_name = None x = Conv2D(nb_filter,kernel_size,padding=padding,strides=strides,activation='relu',name=conv_name)(x) x = BatchNormalization(axis=3,name=bn_name)(x) return x def Conv_Block(inpt,nb_filter,kernel_size,strides=(1,1), with_conv_shortcut=False): x = Conv2d_BN(inpt,nb_filter=nb_filter[0],kernel_size=(1,1),strides=strides,padding='same') x = Conv2d_BN(x, nb_filter=nb_filter[1], kernel_size=(3,3), padding='same') x = Conv2d_BN(x, nb_filter=nb_filter[2], kernel_size=(1,1), padding='same') if with_conv_shortcut: shortcut = Conv2d_BN(inpt,nb_filter=nb_filter[2],strides=strides,kernel_size=kernel_size) x = add([x,shortcut]) return x else: x = add([x,inpt]) return x def ResNet50(): inpt = Input(shape=(224,224,3)) x = ZeroPadding2D((3,3))(inpt) x = Conv2d_BN(x,nb_filter=64,kernel_size=(7,7),strides=(2,2),padding='valid') x = MaxPooling2D(pool_size=(3,3),strides=(2,2),padding='same')(x) x = Conv_Block(x,nb_filter=[64,64,256],kernel_size=(3,3),strides=(1,1),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[64,64,256],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[64,64,256],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3),strides=(2,2),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[128,128,512],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3),strides=(2,2),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[256,256,1024],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[512,512,2048],kernel_size=(3,3),strides=(2,2),with_conv_shortcut=True) x = Conv_Block(x,nb_filter=[512,512,2048],kernel_size=(3,3)) x = Conv_Block(x,nb_filter=[512,512,2048],kernel_size=(3,3)) x = AveragePooling2D(pool_size=(7,7))(x) x = Flatten()(x) x = Dense(1000,activation='softmax')(x) model = Model(inputs=inpt,outputs=x) return model

• DenseNet-121的Keras實現(xiàn)：

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160

  def DenseNet121(nb_dense_block=4, growth_rate=32, nb_filter=64, reduction=0.0, dropout_rate=0.0, weight_decay=1e-4, classes=1000, weights_path=None): '''Instantiate the DenseNet 121 architecture, # Arguments nb_dense_block: number of dense blocks to add to end growth_rate: number of filters to add per dense block nb_filter: initial number of filters reduction: reduction factor of transition blocks. dropout_rate: dropout rate weight_decay: weight decay factor classes: optional number of classes to classify images weights_path: path to pre-trained weights # Returns A Keras model instance. ''' eps = 1.1e-5 # compute compression factor compression = 1.0 - reduction # Handle Dimension Ordering for different backends global concat_axis if K.image_dim_ordering() == 'tf': concat_axis = 3 img_input = Input(shape=(224, 224, 3), name='data') else: concat_axis = 1 img_input = Input(shape=(3, 224, 224), name='data') # From architecture for ImageNet (Table 1 in the paper) nb_filter = 64 nb_layers = [6,12,24,16] # For DenseNet-121 # Initial convolution x = ZeroPadding2D((3, 3), name='conv1_zeropadding')(img_input) x = Convolution2D(nb_filter, 7, 7, subsample=(2, 2), name='conv1', bias=False)(x) x = BatchNormalization(epsilon=eps, axis=concat_axis, name='conv1_bn')(x) x = Scale(axis=concat_axis, name='conv1_scale')(x) x = Activation('relu', name='relu1')(x) x = ZeroPadding2D((1, 1), name='pool1_zeropadding')(x) x = MaxPooling2D((3, 3), strides=(2, 2), name='pool1')(x) # Add dense blocks for block_idx in range(nb_dense_block - 1): stage = block_idx+2 x, nb_filter = dense_block(x, stage, nb_layers[block_idx], nb_filter, growth_rate, dropout_rate=dropout_rate, weight_decay=weight_decay) # Add transition_block x = transition_block(x, stage, nb_filter, compression=compression, dropout_rate=dropout_rate, weight_decay=weight_decay) nb_filter = int(nb_filter * compression) final_stage = stage + 1 x, nb_filter = dense_block(x, final_stage, nb_layers[-1], nb_filter, growth_rate, dropout_rate=dropout_rate, weight_decay=weight_decay) x = BatchNormalization(epsilon=eps, axis=concat_axis, name='conv'+str(final_stage)+'_blk_bn')(x) x = Scale(axis=concat_axis, name='conv'+str(final_stage)+'_blk_scale')(x) x = Activation('relu', name='relu'+str(final_stage)+'_blk')(x) x = GlobalAveragePooling2D(name='pool'+str(final_stage))(x) x = Dense(classes, name='fc6')(x) x = Activation('softmax', name='prob')(x) model = Model(img_input, x, name='densenet') if weights_path is not None: model.load_weights(weights_path) return model def conv_block(x, stage, branch, nb_filter, dropout_rate=None, weight_decay=1e-4): '''Apply BatchNorm, Relu, bottleneck 1x1 Conv2D, 3x3 Conv2D, and option dropout # Arguments x: input tensor stage: index for dense block branch: layer index within each dense block nb_filter: number of filters dropout_rate: dropout rate weight_decay: weight decay factor ''' eps = 1.1e-5 conv_name_base = 'conv' + str(stage) + '_' + str(branch) relu_name_base = 'relu' + str(stage) + '_' + str(branch) # 1x1 Convolution (Bottleneck layer) inter_channel = nb_filter * 4 x = BatchNormalization(epsilon=eps, axis=concat_axis, name=conv_name_base+'_x1_bn')(x) x = Scale(axis=concat_axis, name=conv_name_base+'_x1_scale')(x) x = Activation('relu', name=relu_name_base+'_x1')(x) x = Convolution2D(inter_channel, 1, 1, name=conv_name_base+'_x1', bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) # 3x3 Convolution x = BatchNormalization(epsilon=eps, axis=concat_axis, name=conv_name_base+'_x2_bn')(x) x = Scale(axis=concat_axis, name=conv_name_base+'_x2_scale')(x) x = Activation('relu', name=relu_name_base+'_x2')(x) x = ZeroPadding2D((1, 1), name=conv_name_base+'_x2_zeropadding')(x) x = Convolution2D(nb_filter, 3, 3, name=conv_name_base+'_x2', bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) return x def transition_block(x, stage, nb_filter, compression=1.0, dropout_rate=None, weight_decay=1E-4): ''' Apply BatchNorm, 1x1 Convolution, averagePooling, optional compression, dropout # Arguments x: input tensor stage: index for dense block nb_filter: number of filters compression: calculated as 1 - reduction. Reduces the number of feature maps in the transition block. dropout_rate: dropout rate weight_decay: weight decay factor ''' eps = 1.1e-5 conv_name_base = 'conv' + str(stage) + '_blk' relu_name_base = 'relu' + str(stage) + '_blk' pool_name_base = 'pool' + str(stage) x = BatchNormalization(epsilon=eps, axis=concat_axis, name=conv_name_base+'_bn')(x) x = Scale(axis=concat_axis, name=conv_name_base+'_scale')(x) x = Activation('relu', name=relu_name_base)(x) x = Convolution2D(int(nb_filter * compression), 1, 1, name=conv_name_base, bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) x = AveragePooling2D((2, 2), strides=(2, 2), name=pool_name_base)(x) return x def dense_block(x, stage, nb_layers, nb_filter, growth_rate, dropout_rate=None, weight_decay=1e-4, grow_nb_filters=True): ''' Build a dense_block where the output of each conv_block is fed to subsequent ones # Arguments x: input tensor stage: index for dense block nb_layers: the number of layers of conv_block to append to the model. nb_filter: number of filters growth_rate: growth rate dropout_rate: dropout rate weight_decay: weight decay factor grow_nb_filters: flag to decide to allow number of filters to grow ''' eps = 1.1e-5 concat_feat = x for i in range(nb_layers): branch = i+1 x = conv_block(concat_feat, stage, branch, growth_rate, dropout_rate, weight_decay) concat_feat = merge([concat_feat, x], mode='concat', concat_axis=concat_axis, name='concat_'+str(stage)+'_'+str(branch)) if grow_nb_filters: nb_filter += growth_rate return concat_feat, nb_filter

總結

以上是生活随笔為你收集整理的经典的卷积神经网络简介的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。