情感分析：基于循環神經網絡

Sentiment Analysis: Using Recurrent Neural Networks

與搜索同義詞和類比詞類似，文本分類也是單詞嵌入的一個下游應用。在本文中，將應用預訓練的詞向量（glow）和具有多個隱藏層的雙向遞歸神經網絡，如圖1所示。將使用該模型來判斷長度不定的文本序列是包含積極情緒還是消極情緒。

圖1. 本節將經過預訓練的GloVe to RNN-based提供給基于RNN的體系結構，用于情感分析。

from d2l import mxnet as d2l

from mxnet import gluon, init, np, npx

from mxnet.gluon import nn, rnn

npx.set_np()

batch_size = 64

train_iter, test_iter, vocab = d2l.load_data_imdb(batch_size)

1. Using a Recurrent Neural Network Model

在該模型中，每個詞首先從嵌入層獲得一個特征向量。然后，利用雙向遞歸神經網絡對特征序列進行編碼，得到序列信息。最后，將編碼后的序列信息通過全連通層輸出。具體來說，可以將雙向長短期存儲器的隱藏狀態連接在初始時間步和最終時間步中，并將其作為編碼的特征序列信息傳遞給輸出層分類。在下面實現的BiRNN類中，嵌入實例是嵌入層，LSTM實例是序列編碼的隱藏層，密集實例是生成分類結果的輸出層。

class BiRNN(nn.Block):

def __init__(self, vocab_size, embed_size, num_hiddens,num_layers, **kwargs):super(BiRNN, self).__init__(**kwargs)self.embedding = nn.Embedding(vocab_size, embed_size)# Set Bidirectional to True to get a bidirectional recurrent neural# networkself.encoder = rnn.LSTM(num_hiddens, num_layers=num_layers,bidirectional=True, input_size=embed_size)self.decoder = nn.Dense(2)def forward(self, inputs):# The shape of inputs is (batch size, number of words). Because LSTM# needs to use sequence as the first dimension, the input is# transformed and the word feature is then extracted. The output shape# is (number of words, batch size, word vector dimension).embeddings = self.embedding(inputs.T)# Since the input (embeddings) is the only argument passed into# rnn.LSTM, it only returns the hidden states of the last hidden layer# at different timestep (outputs). The shape of outputs is# (number of words, batch size, 2 * number of hidden units).outputs = self.encoder(embeddings)# Concatenate the hidden states of the initial timestep and final# timestep to use as the input of the fully connected layer. Its# shape is (batch size, 4 * number of hidden units)encoding = np.concatenate((outputs[0], outputs[-1]), axis=1)outs = self.decoder(encoding)return outs

創建一個具有兩個隱藏層的雙向遞歸神經網絡。

embed_size, num_hiddens, num_layers, ctx = 100, 100, 2, d2l.try_all_gpus()

net = BiRNN(len(vocab), embed_size, num_hiddens, num_layers)

net.initialize(init.Xavier(), ctx=ctx)

1.1. Loading Pre-trained Word Vectors

由于用于情感分類的訓練數據集不是很大，為了處理過擬合，將直接使用在較大語料庫上預先訓練的詞向量作為所有詞的特征向量。這里，為字典vocab中的每個單詞加載一個100維的GloVe詞向量。

glove_embedding = d2l.TokenEmbedding(‘glove.6b.100d’)

查詢詞匯表中的單詞向量。

embeds = glove_embedding[vocab.idx_to_token]

embeds.shape

(49339, 100)

然后，將這些詞向量作為評論中每個詞的特征向量。請注意，預先訓練的詞向量的維數需要與創建的模型中的嵌入層輸出大小embed_size一致。此外，不再在訓練期間更新這些詞向量。

net.embedding.weight.set_data(embeds)

net.embedding.collect_params().setattr(‘grad_req’, ‘null’)

2. Training and Evaluating the Model

現在，可以開始訓練了。

lr, num_epochs = 0.01, 5

trainer = gluon.Trainer(net.collect_params(), ‘adam’, {‘learning_rate’: lr})

loss = gluon.loss.SoftmaxCrossEntropyLoss()

d2l.train_ch13(net, train_iter, test_iter, loss, trainer, num_epochs, ctx)

loss 0.299, train acc 0.872, test acc 0.842

626.7 examples/sec on [gpu(0), gpu(1)]

最后，定義了預測函數。

#@save

def predict_sentiment(net, vocab, sentence):

sentence = np.array(vocab[sentence.split()], ctx=d2l.try_gpu())label = np.argmax(net(sentence.reshape(1, -1)), axis=1)

return ‘positive’ if label == 1 else ‘negative’

然后，利用訓練好的模型對兩個簡單句子的情感進行分類。

predict_sentiment(net, vocab, ‘this movie is so great’)

‘positive’

predict_sentiment(net, vocab, ‘this movie is so bad’)

‘negative’

2. Summary

· Text classification transforms a sequence of text of indefinite length into a category of text. This is a downstream application of word embedding.

· We can apply pre-trained word vectors and recurrent neural networks to classify the emotions in a text.

總結

以上是生活随笔為你收集整理的情感分析：基于循环神经网络的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。