貓狗大戰是kaggle平臺上的一個比賽，用于實現貓和狗的二分類問題。最近在學卷積神經網絡，所以自己動手搭建了幾層網絡進行訓練，然后利用遷移學習把別人訓練好的模型直接應用于貓狗分類這個數據集，比較一下實驗效果。

自己搭建網絡

需要用到的庫

import tensorflow as tf from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D from tensorflow.keras.preprocessing.image import ImageDataGenerator import os import numpy as np import matplotlib.pyplot as plt

數據集加載
數據是通過這個網站下載的，也可以自己先下載好。

dataset_url = "https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip"dataset_path = tf.keras.utils.get_file("cats_and_dogs_filtered.zip", origin=dataset_url, extract=True) dataset_dir = os.path.join(os.path.dirname(dataset_path), "cats_and_dogs_filtered")train_cats = os.path.join(dataset_dir,"train","cats") train_dogs = os.path.join(dataset_dir,"train","dogs")test_cats = os.path.join(dataset_dir,"validation","cats") test_dogs = os.path.join(dataset_dir,"validation","dogs")train_dir = os.path.join(dataset_dir,"train") test_dir = os.path.join(dataset_dir,"validation")

統計訓練集和測試集的大小

train_dogs_num = len(os.listdir(train_dogs)) train_cats_num = len(os.listdir(train_cats))test_dogs_num = len(os.listdir(test_dogs)) test_cats_num = len(os.listdir(test_cats))train_all = train_cats_num+train_dogs_num test_all = test_cats_num+test_dogs_num

設置超參數

batch_size = 128 epochs = 50 height = 150 width = 150

數據預處理
我們所作的預處理包含以下幾步：
①讀取圖像數據。
②對圖像內容進行解碼并轉換成合適的格式。
③對圖像進行打散、規定圖片大小。
④將數值歸一化。

train_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1.0/255) test_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1.0/255) train_data_gen = train_generator.flow_from_directory(batch_size=batch_size,directory=train_dir,shuffle=True,target_size=(height,width),class_mode="binary") test_data_gen = train_generator.flow_from_directory(batch_size=batch_size,directory=test_dir,shuffle=True,target_size=(height,width),class_mode="binary")

構建網絡
網絡模型為：3層卷積池化層+Dropout+Flatten+兩層全連接層

model = tf.keras.Sequential([tf.keras.layers.Conv2D(16,3,padding="same",activation="relu",input_shape=(height,width,3)),tf.keras.layers.MaxPool2D(),tf.keras.layers.Conv2D(32,3,padding="same",activation="relu"),tf.keras.layers.MaxPool2D(),tf.keras.layers.Conv2D(64,3,padding="same",activation="relu"),tf.keras.layers.MaxPool2D(),tf.keras.layers.Dropout(0.5),tf.keras.layers.Flatten(),tf.keras.layers.Dense(512,activation="relu"),tf.keras.layers.Dense(1) ]) model.compile(optimizer="adam",loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),metrics=["acc"])

訓練模型

history = model.fit_generator(train_data_gen,steps_per_epoch=train_all//batch_size,epochs=epochs,validation_data=test_data_gen,validation_steps=test_all//batch_size)

訓練結果可視化

#訓練結果可視化 accuracy = history.history["acc"] test_accuracy = history.history["val_acc"] loss = history.history["loss"] test_loss = history.history["val_loss"] epochs_range = range(epochs) plt.figure(figsize=(10,5)) plt.subplot(1,2,1) plt.plot(epochs_range,accuracy,label = "Training Acc") plt.plot(epochs_range,test_accuracy,label = "Test Acc") plt.legend() plt.title("Training and Test Acc")plt.subplot(1,2,2) plt.plot(epochs_range,loss,label = "Training loss") plt.plot(epochs_range,test_loss,label = "Test loss") plt.legend() plt.title("Training and Test loss") plt.show()

自己搭建的網絡對貓狗大戰的數據進行訓練，經過50次的epoch，最終的訓練結果是70%左右的正確率，不是很高。
訓練結果：
其中訓練集的模型準確率接近100%，但是測試集的正確率比較低。

Epoch 50/501/15 [=>............................] - ETA: 5s - loss: 0.0089 - acc: 1.00002/15 [===>..........................] - ETA: 3s - loss: 0.0071 - acc: 1.00003/15 [=====>........................] - ETA: 4s - loss: 0.0086 - acc: 1.00004/15 [=======>......................] - ETA: 3s - loss: 0.0113 - acc: 0.99785/15 [=========>....................] - ETA: 3s - loss: 0.0163 - acc: 0.99666/15 [===========>..................] - ETA: 3s - loss: 0.0163 - acc: 0.99587/15 [=============>................] - ETA: 2s - loss: 0.0149 - acc: 0.99658/15 [===============>..............] - ETA: 2s - loss: 0.0135 - acc: 0.99699/15 [=================>............] - ETA: 2s - loss: 0.0139 - acc: 0.9964 10/15 [===================>..........] - ETA: 1s - loss: 0.0145 - acc: 0.9959 11/15 [=====================>........] - ETA: 1s - loss: 0.0139 - acc: 0.9963 12/15 [=======================>......] - ETA: 1s - loss: 0.0155 - acc: 0.9953 13/15 [=========================>....] - ETA: 0s - loss: 0.0155 - acc: 0.9944 14/15 [===========================>..] - ETA: 0s - loss: 0.0170 - acc: 0.9943 15/15 [==============================] - 9s 595ms/step - loss: 0.0174 - acc: 0.9941 - val_loss: 1.2763 - val_acc: 0.7522

經過分析可得，出現了Overfitting的情況，我們對數據集做些調整。
對原來的數據集，做隨機翻轉，水平翻轉，隨機放大操作

train_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1.0/255,rotation_range=45,#隨機翻轉width_shift_range=.15,height_shift_range=.15,horizontal_flip=True,#水平翻轉zoom_range=0.5#放大操作)

最終的訓練結果為：

Epoch 50/501/15 [=>............................] - ETA: 5s - loss: 0.5440 - acc: 0.70312/15 [===>..........................] - ETA: 9s - loss: 0.5151 - acc: 0.74223/15 [=====>........................] - ETA: 10s - loss: 0.5249 - acc: 0.70834/15 [=======>......................] - ETA: 10s - loss: 0.5082 - acc: 0.72275/15 [=========>....................] - ETA: 8s - loss: 0.4787 - acc: 0.7382 6/15 [===========>..................] - ETA: 8s - loss: 0.4764 - acc: 0.74177/15 [=============>................] - ETA: 7s - loss: 0.4801 - acc: 0.74068/15 [===============>..............] - ETA: 6s - loss: 0.4759 - acc: 0.75319/15 [=================>............] - ETA: 5s - loss: 0.4805 - acc: 0.7554 10/15 [===================>..........] - ETA: 4s - loss: 0.4877 - acc: 0.7459 11/15 [=====================>........] - ETA: 3s - loss: 0.4944 - acc: 0.7390 12/15 [=======================>......] - ETA: 2s - loss: 0.4969 - acc: 0.7325 13/15 [=========================>....] - ETA: 1s - loss: 0.4939 - acc: 0.7345 14/15 [===========================>..] - ETA: 0s - loss: 0.4933 - acc: 0.7368 15/15 [==============================] - 23s 2s/step - loss: 0.4957 - acc: 0.7377 - val_loss: 0.5394 - val_acc: 0.7277

Overfitting的情況得到了改善，但是準確率沒有得到相應的提高。這是經過50個epoch之后的結果。

遷移學習

所謂的遷移學習就是通過別人已經訓練好的網絡直接對自己的數據進行處理。所采用的網絡是別人已經訓練好的VGG16網絡。

模型加載

#引用VGG16模型 conv_base = tf.keras.applications.VGG16(weights='imagenet',include_top=False) #設置為不可訓練 conv_base.trainable =False #模型搭建 model = tf.keras.Sequential() model.add(conv_base) model.add(tf.keras.layers.GlobalAveragePooling2D()) model.add(tf.keras.layers.Dense(512,activation='relu')) model.add(tf.keras.layers.Dense(1,activation='sigmoid'))

模型訓練

model.compile(optimizer='Adam',loss='binary_crossentropy',metrics=['acc']) history = model.fit(train_data_gen,epochs=epochs,steps_per_epoch=train_all//batch_size,validation_data=test_data_gen,validation_steps=test_all//batch_size)

訓練結果

Epoch 10/101/15 [=>............................] - ETA: 4:36 - loss: 0.3333 - acc: 0.84382/15 [===>..........................] - ETA: 2:14 - loss: 0.3917 - acc: 0.81253/15 [=====>........................] - ETA: 1:25 - loss: 0.3870 - acc: 0.81554/15 [=======>......................] - ETA: 1:01 - loss: 0.3967 - acc: 0.80825/15 [=========>....................] - ETA: 46s - loss: 0.3994 - acc: 0.8125 6/15 [===========>..................] - ETA: 36s - loss: 0.3935 - acc: 0.81397/15 [=============>................] - ETA: 28s - loss: 0.3976 - acc: 0.81608/15 [===============>..............] - ETA: 22s - loss: 0.3948 - acc: 0.81669/15 [=================>............] - ETA: 17s - loss: 0.3958 - acc: 0.8207 10/15 [===================>..........] - ETA: 13s - loss: 0.3970 - acc: 0.8198 11/15 [=====================>........] - ETA: 10s - loss: 0.3934 - acc: 0.8243 12/15 [=======================>......] - ETA: 7s - loss: 0.3946 - acc: 0.8246 13/15 [=========================>....] - ETA: 4s - loss: 0.3880 - acc: 0.8274 14/15 [===========================>..] - ETA: 2s - loss: 0.3900 - acc: 0.8251 15/15 [==============================] - 48s 3s/step - loss: 0.3910 - acc: 0.8237 - val_loss: 0.4328 - val_acc: 0.8025

因為VGG16模型訓練起來比較耗時，所以我只設置了10個epoch，但是最終的結果已經比自己搭建的網絡好很多了。

沒有出現過擬合的情況（數據集經過預處理了），而且在epoch只有10的情況下，正確率已經達到了80%。

總結

通過對比我們可以發現， 自己搭建的網絡在模型準確率上面，不如遷移學習所使用的網絡模型，有可能是我的網絡泛化能力比較差的問題。路過的大佬如果有更好的網絡模型，可以討論一下。

參考博客1

參考博客2

總結

以上是生活随笔為你收集整理的基于tensorflow2.0实现猫狗大战（搭建网络迁移学习）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。