【論文閱讀】The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification

• 摘要
• 具體實現
• • 判別組件
  • 差異性組件
• 代碼

摘要

Key for solving fine-grained image categorization is finding discriminate and local regions that correspond to subtle visual traits. Great strides have been made, with complex networks designed specifically to learn part-level discriminate feature representations. In this paper, we show it is possible to cultivate subtle details without the need for overly complicated network designs or training mechanisms – a single loss is all it takes. The main trick lies with how we delve into individual feature channels early on, as opposed to the convention of starting from a consolidated feature map. The proposed loss function, termed as mutual-channel loss (MC-Loss), consists of two channel-specific components: a discriminality component and a diversity component. The discriminality component forces all feature channels belonging to the same class to be discriminative, through a novel channel-wise attention mechanism. The diversity component additionally constraints channels so that they become mutually exclusive on spatial-wise. The end result is therefore a set of feature channels that each reflects different locally discriminative regions for a specific class. The MC-Loss can be trained end-to-end, without the need for any bounding-box/part annotations, and yields highly discriminative regions during inference. Experimental results show our MC-Loss when implemented on top of common base networks can achieve state-of-the-art performance on all four fine-grained categorization datasets (CUB-Birds, FGVC-Aircraft, Flowers-102, and Stanford-Cars). Ablative studies further demonstrate the superiority of MC-Loss when compared with other recently proposed general-purpose losses for visual classification, on two different base networks. Code available at https://github.com/dongliangchang/Mutual-Channel-Loss

解決細粒度圖像分類的關鍵是找到與細微視覺特征相對應的區分區域和局部區域。復雜的網絡專門設計用來學習零件級的區別特征表示已經取得了長足的進步。在這篇文章中，我們展示了在不需要過于復雜的網絡設計或訓練機制的情況下培養微妙的細節是可能的——只需一個損失就可以了。主要的訣竅在于我們如何在早期深入研究各個特征通道，而不是從統一的特征圖開始的慣例。所提出的損失函數稱為互通道損失(MC-Loss)，由兩個特定通道的組件組成：判別性組件和差異性組件。判別性組件通過一種新穎的通道注意力機制，強制屬于同一類別的所有特征頻道具有鑒別性。差異性組件另外約束通道，使得它們在空間上變得相互排斥。因此，最終結果是一組特征通道，每個通道反映特定類別的不同局部區分區域。MC-Loss可以端到端訓練，不需要任何額外的bounding box/part標注，并在推理過程中產生高度可分辨的區域。實驗結果表明，當我們的MCLoss在公共基礎網絡上實現時，可以在所有四個細粒度分類數據集(Cub-Birds、FGVC-Aircraft、Flowers102和Stanford-Cars)上獲得最先進的性能。消融研究進一步證明，在兩種不同的基礎網絡上，MC-Loss與最近提出的其他用于視覺分類的通用損失相比具有優越性。代碼可在https://github.com/dongliangchang/Mutual-Channel-Loss獲得

具體實現

———————————————————————————————————————
將圖像輸入到基本網絡提取特征得到$\mathcal{F}\in R^{N\times W\times H}$，其中需要將通道$N$設置成$c\times \xi$，其中，$c$為類別數量，$\xi$為平坦到每個類上的通道數。
由此第i個類別的特征可以表示為：
$${\mathbf{F}}_i=\left\{{\mathcal{F}}_{i\times \xi +1},{\mathcal{F}}_{i\times \xi +2},?,{\mathcal{F}}_{i\times \xi +\xi }\right\}$$
之前提取的特征就可以表示為：
$$\mathbf{F}=\left\{{\mathbf{F}}_0,{\mathbf{F}}_1,?,{\mathbf{F}}_{c?1}\right\}$$
隨后，$F$進入網絡的兩個流，其中兩個不同的損失針對兩個不同的目標定制。在上圖中，$L_{CE}$流將$F$視為輸入到全連接層中，作交叉熵損失，在這里，交叉熵損失鼓勵網絡提取主要集中在全局判別區域的信息特征。$L_{MC}$流監督網絡聚焦不同的局部區分性區域，對$L_{MC}$做一個加權得到總的損失：
$$\mathrm{Loss}(\mathbf{F})=L_{CE}(\mathbf{F})+\mu \times L_{MC}(\mathbf{F})$$

再來具體看$L_{MC}$流：

判別組件

上圖（a）左，根據前面劃分的$\mathbf{F}=\left\{{\mathbf{F}}_0,{\mathbf{F}}_1,?,{\mathbf{F}}_{c?1}\right\}$，判別組件要求特征通道與類對齊，每個對應于特定類的特征通道都應該具有足夠的判別能力。
$$L_{dis}(\mathbf{F})=L_{CE}(\mathbf{y},\underset{\text{Softmax?}}{\frac{[e^{g\left({\mathbf{F}}_0\right)},e^{g\left({\mathbf{F}}_1\right)},?,e^{{g\left({\mathbf{F}}_{c?1}\right)]}^{\mathrm{T}}}}{{\sum }_{i=0}^{c?1}{e}^{g\left({\mathbf{F}}_i\right)}}})$$
其中，$g\left({\mathbf{F}}_i\right)=\underset{\text{GAP?}}{\frac{1}{WH}\sum _{k=1}^{WH}}\underset{\text{CCMP?}}{\underset{j=1,2,?,\xi }{\max }}\underset{\text{CWA?}}{\left[M_i?{\mathbf{F}}_{i,j,k}\right]}$；$M_i=\mathrm{diag}($ Mask ${}_i)$；Mask ${}_i\in R^{\xi }$ is a 0-1 mask with randomly $?\frac{\xi }{2}?$zero(s)；GAP，CCMP，CWA，分別為Global Average Pooling，Cross-Channel Max Pooling，Channel-Wise Attention。

差異性組件

差異性組件是用于所有特征通道的近似距離測量，以計算所有通道的總相似度。與歐幾里德距離和二次復雜度的Kullback-Leibler散度等常用度量相比，在計算復雜度不變的情況下，它的計算成本更低。沿著上圖(a)的右側塊所示的差異性組件讓組$L_i$中的特征通道通過訓練變得彼此不同。換言之，一個類別的不同特征通道應該聚焦于圖像的不同區域，而不是所有通道都聚焦于最具區分性的區域。因此，它通過使每個組的特征通道多樣化來減少冗余信息，并有助于發現針對圖像中每一類的不同區分區域。該操作可以被解釋為跨通道去相關，以便從圖像的不同顯著區域捕捉細節。在Softmax之后，通過引入CCMP直接在卷積濾波器上施加監督，然后進行空間維度求和來衡量相交程度。$L_{div}$表示為：
$$L_{div}(\mathbf{F})=\frac{1}{c}\sum _{i=0}^{c?1}h\left({\mathbf{F}}_i\right)$$
其中，$h\left({\mathbf{F}}_i\right)={\sum }_{k=1}^{WH}\underset{CCMP}{\underset{j=1,2,?,\xi }{\max }}\underset{\text{Softmax?}}{\left[\frac{e^{{\mathbf{F}}_{i,j,k}}}{{\sum }_{k^{\prime }=1}^{WH}{e}^{{\mathbf{F}}_{i,j,k^{\prime }}}}\right]}$

代碼

def Mask(nb_batch, channels):foo = [1] * 2 + [0] * 1bar = []for i in range(200):random.shuffle(foo)bar += foobar = [bar for i in range(nb_batch)]bar = np.array(bar).astype("float32")bar = bar.reshape(nb_batch,200*channels,1,1)bar = torch.from_numpy(bar)bar = bar.cuda()bar = Variable(bar)return bardef supervisor(x,targets,height,cnum):mask = Mask(x.size(0), cnum)branch = xbranch = branch.reshape(branch.size(0),branch.size(1), branch.size(2) * branch.size(3))branch = F.softmax(branch,2)branch = branch.reshape(branch.size(0),branch.size(1), x.size(2), x.size(2))branch = my_MaxPool2d(kernel_size=(1,cnum), stride=(1,cnum))(branch) branch = branch.reshape(branch.size(0),branch.size(1), branch.size(2) * branch.size(3))loss_2 = 1.0 - 1.0*torch.mean(torch.sum(branch,2))/cnum # set margin = 3.0branch_1 = x * mask branch_1 = my_MaxPool2d(kernel_size=(1,cnum), stride=(1,cnum))(branch_1) branch_1 = nn.AvgPool2d(kernel_size=(height,height))(branch_1)branch_1 = branch_1.view(branch_1.size(0), -1)loss_1 = criterion(branch_1, targets)return [loss_1, loss_2] class model_bn(nn.Module):def __init__(self, feature_size=512,classes_num=200):super(model_bn, self).__init__() self.features = nn.Sequential(*list(net.children())[:-2]) self.max = nn.MaxPool2d(kernel_size=14, stride=14)self.num_ftrs = 600*1*1self.classifier = nn.Sequential(nn.BatchNorm1d(self.num_ftrs),#nn.Dropout(0.5),nn.Linear(self.num_ftrs, feature_size),nn.BatchNorm1d(feature_size),nn.ELU(inplace=True),#nn.Dropout(0.5),nn.Linear(feature_size, classes_num),)def forward(self, x, targets):x = self.features(x)if self.training:MC_loss = supervisor(x,targets,height=14,cnum=3)x = self.max(x)x = x.view(x.size(0), -1)x = self.classifier(x)loss = criterion(x, targets)if self.training:return x, loss, MC_losselse:return x, loss

train

out, ce_loss, MC_loss = net(inputs, targets)loss = ce_loss + args["alpha_1"] * MC_loss[0] + args["beta_1"] * MC_loss[1]

valid

out, ce_loss = net(inputs,targets)test_loss += ce_loss.item()

總結

以上是生活随笔為你收集整理的【论文阅读】The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。