A novel sentiment analysis method based on multi-scale deep learning

1.   Introduction

The rapid development of newly emerging techniques such as computer, mobile computing and big data has facilitated the arrival of new media era ^[1,2]. As the native of mobile Internet, college students have strong sense in pursuit of free voice, as well as strong willingness in discussion of public topics ^[3,4]. Nowadays, they have been one of the most active user groups in mobile social media ^[5,6]. But impacted by increasingly fruitful multiple information channels, the information monopoly of mainstream media has been challenged unprecedentedly ^[7,8]. Various negative and false information on the Internet are mixed with massive information, easily bringing emotional fluctuation to college students ^[9,10]. Among them, the more typical is the impact of film and television works ^[11]. At present, the variety of film and television works on the Internet is complex, which has a positive or negative impact on college students' life attitude to a certain extent ^[12]. This circumstance may even influence mental health of college students and harmony of campuses ^[13,14]. Therefore, effective sentiment analysis for campus public opinions in mobile social media ^[15], is of great importance to ideological management in colleges and universities ^[16].

During the past few years, the wide application of deep neural networks have been continuously explored in terms of its powerful information processing ability ^[17], and have also been the mainstream technology for sentiment analysis ^[18]. The most popular deep sentiment analysis methods are established upon recurrent neural networks (RNN) ^[19]. The RNN emphasizes the dependency among word-building units of text sequences, and builds a global feature representation on this basis ^[20]. From the perspective of linguistics, there is often some contextual dependency among the characters of text, which is more consistent with the RNN-based methods in scenarios ^[21]. Typical models include long short-term memory (LSTM) and its bidirectional version ^[22]. However, existing methods only considered the vertical features of the text, yet ignoring the potential horizontal features. From the perspective of model structure, they concentrated more on learning depth than learning width to some extent ^[23,24].

In order to make up the aforementioned gap, this paper extends the RNN-based models from a single scale to multi-dimensional scales. Keeping the depth stable, the number of concurrency to increase the learning fields is increased ^[25]. It is expected to get a more comprehensive semantic feature expression to handle the imbalance problem between model depth and model width. Therefore, taking English text based campus public opinion as the object, this paper proposes a novel sentiment analysis method based on multi-scale deep learning for college public opinions (named as MSLSTM-CPO for short). In order to match the bidirectional dependency of semantic sequences, Bi-LSTM is selected as the basic networking unit. Vertically, a single word in English is used as the minimum processing unit. Horizontally, more than two basic networking units are adopted to form a parallel computing structure. As a result, the model has been enhanced with respect to horizonal layer numbers, realizing the trade-off between depth and breadth. We can summarize major working points of this paper as following aspects:

● This work discusses and demonstrates semantic characteristics from both vertical direction and horizonal direction.

● This paper develops, MSLSTM-CPO, a novel sentiment analysis method via multi-scale deep learning for college public opinions.

● This work conducts some experiments to comprehensively assess performance of the proposal.

2.   Related works

In recent years, sentiment analysis has received more and more attention as an important research direction in the field of natural language processing, and has gradually become a research hotspot. From the analysis of its research development history, sentiment analysis has mainly gone through three stages: lexicon based method, machine learning method and deep learning method.

In the initial stages of sentiment analysis, lexicon-based statistics on the number of sentiment subscripts was the main method. It is obvious that lexicon-based statistical methods, although easy to understand, do not develop generalisation very well. With the development of natural language processing, machine learning methods research has so far also yielded many effective results ^[26,27,28]. Rashmi et al. ^[29] proposed a soft voting classifier by integrating five baseline models of logistic regression, balanced random forest, eXtreme Gradient Boosting, random forest, and support vector machine with the task goal of classifying mixed Indian languages. The method is useful for classifying positive, negative, neutral, mixed emotions and unemotional states with better results. Maipraditj et al. ^[30] proposed a machine learning-based approach that uses three different datasets for text processing, using the N-gram IDF method for feature extraction, and then utilizes automated machine learning to classify positive, neutral emotions, and negative emotions. However, traditional machine learning methods have the drawback of not being able to combine semantic information with textual context, so deep learning is gradually becoming a research trend ^{[31,32,33,34,35,36]}. Huang et al. ^[37] explored changes in emotion in blended learning using LSTM-based text mining methods and epistemic network analysis. Jia et al. ^[38] constructed a sentiment classification model using BERT, CNN and attention mechanism methods to mine text for contextual connections and features. Harendranath et al. ^[39] proposed a modeling method based on recurrent neural network to classify the emotions of political comments.

To sum up, almost all the related works dealt with various semantic analysis problems from the perspective of vertical-directional semantics. Nevertheless, semantics modeling from both vertical and horizonal directions, still needs to be deeply discussed and considered. Therefore, the next section of this paper is going to display the proposed technical framework from such point.

3.   Methodology

3.1.   Overview

Currently, almost all valid natural language models are built on modelling the serialisation of natural utterances. The data for this experiment is English comment text data, and in most cases the semantic information of each comment keyword is closely related to the information of the previous text. Therefore, a neural network that can remember the context can better handle the sentiment analysis of the comment text for this experiment. The traditional LSTM model achieves the ability to remember the text and can handle the problem of internal information complexity and overload caused by long sequential data. In order to match the bidirectional dependency of semantic sequences, Bi-LSTM is chosen as the basic networking unit for this experiment. Hence, a novel sentiment analysis method for college public opinions via multi-scale deep learning is develped in this work. As shown in Figure 1, the whole workflow of the MSLSTM-CPO model consists of two parts: Word-level semantic encoding and Sentence-level semantic encoding.

Figure 1.  The major workflow of the proposed MSLSTM-CPO.

DownLoad: Full-Size Img PowerPoint

3.2.   Word-level semantic encoding

The data for this experiment is English text data and the classification model cannot be trained directly on the input text. Therefore, each English comment text needs to be separated using space characters and then the words are converted into a vector representation. If there are many words in the corpus, this will result in a very high dimensionality of the vector for each word. As a result, a normal one hot encoding would make the word vectors very sparse. Word embedding is a way of representing words in natural language by representing each word as a vector in a high-dimensional space. In this way, natural language computation is converted into vector computation. The detailed steps of the word embedding are shown in Figure 2.

Figure 2.  The workflow diagram for the computation of word embedding.

DownLoad: Full-Size Img PowerPoint

Step 1: Dictionary lookup. The text is first cut into words using spaces to divide long sentences into a number of words. The words in the text sentence are converted into fixed ID integers by querying the dictionary.

Step 2: One-hot encoding. If the dictionary has $P$ words in its word list, each particular word can be represented by a P-dimensional vector, thus converting each ID into a fixed-length vector. For a word with ID $x$, the $xth$ element of the vector is 1 and the rest of the $P-1$ elements are 0. This process is One-hot encoding and can be expressed as:

Step 3: Embedding lookup. In a real-life experimental scenario, each text is of different lengths, both long and short. In order to avoid the overall training result of the model being affected by the length of the data being too long or too short, a parameter max_seq_len is set to truncate and complement the text. After One-hot encoding, the sentence tensor is denoted as $V$. Then, this tensor $V$ is multiplied by another dense tensor $W$, $W \in {\mathbb{R}^{P \times l}}$. $P$ denotes the word table size and $l$ denotes the vector size of each word. After tensor multiplication, this can then be mapped to an embedding representation $X$, thus completing the purpose of representing words as vectors.

3.3.   Sentence-level semantic encoding

Based on word-level semantic encoding, this subsection is sentence-level semantic encoding. For each input text sentence, the sentences are truncated and padded by the set global variable max_seq_len, turning them into fixed-length vectors. The processed information is then trained using the model proposed in this paper to obtain the results of sentiment classification. As shown in Figure 3, the MSLSTM-CPO model proposed in this paper has a total of four layers: embedding layer, LSTM layer, average layer and output layer.

Figure 3.  Schematic diagram of the MSLSTM-CPO network structure.

DownLoad: Full-Size Img PowerPoint

3.3.1.   Bidirectional LSTM layer

The LSTM model cannot process the raw text data directly, and requires Word-Level Semantic Encoding for Word Embedding, which is a vectorised representation of the input text words. Assuming a set of input sequences as ${\mathbf{X}} \in {\mathbb{R}^{B \times L \times M}}$, where $B$ is the batch size, $L$ is the length of the sequence and $M$ is the input feature dimension, the LSTM scans the sequences sequentially from left to right and updates the internal state ${{\mathbf{C}}_t} \in {\mathbb{R}^{B \times D}}$ and the output state ${{\mathbf{H}}_t} \in {\mathbb{R}^{B \times D}}$ of the state at each moment computationally through the loop unit. $D$ denotes the dimensionality of the hidden state vector. The computational steps of the LSTM consist mainly of computing the three gates, computing the internal state and computing the output state.

Step 1: Calculating the three "doors". At moment $t$, the loop unit of the LSTM computes a set of input gates ${{\mathbf{I}}_t}$, oblivion gates ${{\mathbf{F}}_t}$ and output gates ${{\mathbf{O}}_t}$ using the input ${{\mathbf{X}}_t} \in {\mathbb{R}^{B \times M}}$ at the current moment and the output state ${{\mathbf{H}}_{t - 1}} \in {\mathbb{R}^{B \times D}}$ at the previous moment. This experiment uses the paddle framework to build the model, which differs from the conventional LSTM model implemented by itself, with two additional biases and the parameters in front of the input data when the matrix is multiplied. The calculation formulas are as follows:

where ${{\mathbf{W}}_*} \in {\mathbb{R}^{M \times D}}$, ${{\mathbf{U}}_*} \in {\mathbb{R}^{D \times D}}$, ${{\mathbf{b}}_{i*}} \in {\mathbb{R}^{1 \times D}}$, ${{\mathbf{b}}_{h*}} \in {\mathbb{R}^{1 \times D}}$ are learnable parameters and $\sigma$ is a logistic function that controls the values of the "gates" in the ${\text{(0, 1)}}$ interval. The "gates" here are all matrices of ${\text{B}}$ samples, each row being a vector of "gates" of one sample.

Step 2: Calculating internal states. The first step is to calculate the internal state of the candidate with the following equation:

where ${{\mathbf{W}}_{ic}} \in {\mathbb{R}^{M \times D}}, {{\mathbf{U}}_{hc}} \in {\mathbb{R}^{D \times D}}, {{\mathbf{b}}_{ic}} \in {\mathbb{R}^{1 \times D}}, {{\mathbf{b}}_{hc}} \in {\mathbb{R}^{1 \times D}}$ are learnable parameters. Next, the internal state at moment $t$ is then calculated using the use of forgetting gates and input gates, using the following equation:

where $\otimes$ denotes the element-by-element product.

Step 3: Calculating the output state. The current cell state of the LSTM can be calculated according to Eq (3.6) and as follows:

The input of the LSTM cyclic cell structure is the internal state vector ${{\mathbf{C}}_{t - 1}} \in {\mathbb{R}^{B \times D}}$ and the hidden state vector ${{\mathbf{H}}_{t - 1}} \in {\mathbb{R}^{B \times D}}$ at moment $t - 1$, and the output is the state vector ${{\mathbf{C}}_t} \in {\mathbb{R}^{B \times D}}$ and the hidden state vector ${{\mathbf{H}}_t} \in {\mathbb{R}^{B \times D}}$ at the current moment $t$. With the LSTM cyclic cell, the whole network can establish longer distance temporal dependencies, thus solving the difficulty when the sequence data carried by the RNN is too long to be handled. In addition, the LSTM can help the model capture the semantic information of long sentences more fully by selectively ignoring or reinforcing the current memory and input information.

The Paddle framework's built-in LSTM model has several parameters, including direction and num_layers. Direction indicates the direction of the network iteration and can be set to forward or bidirectional, with the default being forward. Num_layers indicates the number of layers in the network and defaults to 1. The multi-layer bidirectional LSTM needs to receive a sequence of vectors to update the cyclic units with forward and reverse respectively. Therefore, it is only necessary to set the parameter direction to bidirectional and num_layers to any desired $n$ when defining the LSTM to use the multi-layer bidirectional LSTM directly.

3.3.2.   Output layer

The average layer is calculated by averaging the hidden states at all positions of the bidirectional LSTM layer and then used as a representation of the whole sentence. In the experiments, the AveragePooling operator is implemented for the aggregation of hidden states. First, the sequence length vector is used to generate a mask matrix, which is used to mask the vectors that fill the placeholder positions in the text sequence. The vectors of the sequence are then summed and averaged.

The final output layer, by using Linear to output the results of the classification. Sentiment analysis is essentially a classification problem and in the practice of classification problems only the logarithmic odds of classification are usually required to be output by the model. First, the Linear layer transforms the last moment of the hidden state vector ${{\mathbf{H}}_L} \in {\mathbb{R}^{B \times D}}$ linearly and then outputs the logits of the classification. The formula is as follows:

where ${{\mathbf{W}}_{out}} \in {\mathbb{R}^{D \times N}}$ and ${{\mathbf{b}}_{out}} \in {\mathbb{R}^N}$ are the learnable weight matrix and bias. $N$ indicates the number of classifications.

4.   Experiments and analysis

4.1.   Dataset and setting

As far as we know, there is no datasets about college public opinions that are publicly available. This work selects a standard dataset IMDB ^[40] in area of sentiment analysis for evaluation. It was a kind of dataset that records short reviews in social media and sentiment information is also associated. The data of IMDB comes from the Internet Movie Database, including the user's comment text and rating information for a movie. In the current Internet era, university students are an inescapable part of the film industry's consumer base. It is easy to see from everyday entertainment life that the majority of the film industry's loyal audience comes from university students who have free time. Movie reviews can be used to determine students' emotions and capture the direction of public opinion in a timely manner, thus guiding students to develop the right values.

The IMDB dataset collected the review information of many films, with a total of 50,000 English review texts. The film viewer can give a score when commenting on the film, ranging from 1–10 points. In the data processing, if the rating is below 5, it is judged as a negative review and the label is 0. If the rating is above 6, it is judged as a positive review and the label is 1. The purpose of this experiment is to determine whether the emotion expressed by users is positive or negative according to the text information of the comments. After 50,000 pieces of data are processed, each sample data includes the user's comment text and 0/1 label for a movie. There are 25,000 training data and 25,000 test data. In the training set and test set, there are 12,500 positive and negative samples respectively.

In the experiments, the MSLSTM-CPO model proposed in this paper is compared with both machine learning methods and deep learning methods, respectively. The machine learning models discussed are SVC, GaussianNB, MultinomialNB, BernoulliNB. The deep learning models are mainly discussed in the MSLSTM-CPO proposed in this paper and other baseline methods: CNN model, LSTM model, and bidirectional LSTM model. The text of the comments in this experiment varied in length, so the max-seq-len parameter was set to 256 to truncate and complement the text. For the other parameters in the model training, batch-size was set to 128, hidden layer size was set to 256 and embedding size was set to 256. Considering the time consumption of model training and test accuracy, epoch was set to 3. The learning rate was set to 0.001, 0.002 and 0.005 respectively. The num_layers of the MSLSTM-CPO model was set to 3. According to the composition of the initial dataset, the split ratio between the training and test sets was 50%. In order to fully test the validity of the experiments, four common evaluation metrics of classification models were used: accuracy, precision, recall and F1-score. Their detailed descriptions can be found from references like ^[2,41] and are left out here.

4.2.   Results and discussion

4.2.1.   Machine learning methods results and discussion

Sentiment analysis is essentially a binary classification problem, so one SVC model and three typical Naive Bayes-based methods were chosen as machine learning comparison models for this experiment. They are named as Gaussian Naive Bayes, Multinomial Naive Bayes, and Bernoulli Naive Bayes, separately. Among them, GaussianNB is Naive Bayes with Gaussian distribution a priori, MultinomialNB is Naive Bayes with polynomial distribution a priori, and BernoulliNB is Naive Bayes with Bernoulli distribution a priori. Machine learning-based sentiment classification models generally have only simple steps: data pre-processing, text vectorisation, and training the classifier.

As shown in Table 1, this experiment computes the four metric values for the five models. The learning rate of the MSLSTM-CPO model is 0.002, and the other parameters are kept constant. From the results, it can be seen that the machine learning method can generally only achieve an accuracy of 0.5 for classification, while the MSLSTM-CPO model can achieve as high as 0.86, which is much higher than the classification effectiveness of the machine learning methods. A visualisation of the evaluation results of MSLSTM-CPO compared to four traditional machine learning models is shown in Figure 6. Its horizontal coordinates indicate the four metric types and the vertical coordinates indicate the magnitude of the values. It is clear from this that the MSLSTM-CPO model, indicated by the blue bars, performs approximately 50% better than the other machine learning models on all four evaluation metrics. Thus, it is again verified that deep learning models are generally able to perform better than traditional machine learning models in terms of sentiment analysis problems.

Table 1.  Experimental results of machine learning models.

Model	Accuracy	F1-score	Recall	Precision
SVC	0.5206	0.4889	0.5275	0.5206
GaussianNB	0.5057	0.4262	0.5075	0.5128
MultinomialNB	0.4936	0.4935	0.4936	0.4936
BernoulliNB	0.5	0.3333	0.5	0.25
MSLSTM-CPO	0.8678	0.8676	0.86755	0.86905

---

 | Show Table

DownLoad: CSV

4.2.2.   Deep learning methods results and discussion

In order to discuss the effectiveness of the MSLSTM-CPO proposed in this paper, three other typical deep learning methods are chosen as comparison models: CNN model, LSTM model, and bidirectional LSTM model.

As shown in Figures 4 and 5, the trends of accuracy and loss in the training phase of the models are indicated for the four methods when the learning rates are set to 0.001, 0.002, 0.003 respectively. For each subplot, the x-axis represents the number of iterative rounds ranging from 1 to 600. The y-axis in Figure 4 represents the value of the accuracy of the model training, and the y-axis in Figure 5 represents the value of the loss of the model training. Each subplot is plotted from 30 consecutive sample points of data, representing the training trend of the same model at different learning rates. It is clear from this that the red curve with the learning rate set to 0.002 shows better results in the training phase of the four models, with the MSLSTM-CPO model showing the most intuitive comparison of trends. As the number of iteration rounds increases, the training accuracy of the four models shows an increasing trend and the training loss shows a decreasing trend, eventually tending to stabilise. This demonstrates that the models are working properly and reasonably well.

Figure 4.  Tendency of four methods with respect to training accracy.

DownLoad: Full-Size Img PowerPoint

Figure 5.  Tendency of four methods with respect to training loss.

DownLoad: Full-Size Img PowerPoint

Figure 6.  The evaluation results of MSLSTM-CPO with four machine learning models.

DownLoad: Full-Size Img PowerPoint

The above trends only represent the normal training of the model, but only a comparative analysis with each evaluation index can reflect the comprehensive expression ability of the model. Therefore, in this experiment, the learning rate was used as the variable to calculate the evaluation index values of the four models MSLSTM-CPO, CNN, LSTM and Bi-LSTM respectively, while the other parameters were set unchanged. As shown in Table 2, the learning rates were set to 0.001, 0.002 and 0.005, respectively. Sentiment analysis is essentially a dichotomous problem, and the commonly used evaluation metrics are accuracy, precision, recall and F1-score. From the cross-sectional data of the table, the evaluation metrics of each model are stable under different parameter settings. From the analysis of the longitudinal data in the table, the MSLSTM-CPO model works best when the learning rate is 0.002, with an ACC of 0.8678, which is 2, 3 and 1.7% higher than CNN, LSTM and Bi-LSTM respectively. The evaluation metrics of the MSLSTM-CPO model were the highest and stable for different learning rate settings. As shown in Figure 7, the representation is a visualisation of the evaluation results of MSLSTM-CPO compared to the other three deep learning models. It has three subfigures that demonstrate such visualization effect from two angles. For Figure 7(a), it demonstrates comparison of four metric values between MSLSTM-CPO and other three methods. For Figure 7(b), (c), they select two typical metrics (Accuracy and F1-score) and display the evaluation results of the four models under three learning rate values (0.001, 0.002 and 0.005). This figure can clearly reflect the fact that the proposed MSLSTM-CPO has better performance results compared with baseline methods.

Table 2.  Experimental results of deep learning models.

Learning rate	Model	Accuracy	F1-score	Recall	Precision
0.001	CNN	0.8565	0.8565	0.8565	0.8566
	LSTM	0.8465	0.84645	0.8465	0.84675
	Bi-LSTM	0.8485	0.8483	0.8487	0.85005
	MSLSTM-CPO	0.8585	0.85815	0.85555	0.85885
0.002	CNN	0.8516	0.8515	0.85165	0.85305
	LSTM	0.8449	0.84485	0.8452	0.84605
	Bi-LSTM	0.8535	0.85345	0.8537	0.8545
	MSLSTM-CPO	0.8678	0.8676	0.86755	0.86905
0.005	CNN	0.83225	0.83255	0.83225	0.8322
	LSTM	0.8411	0.8412	0.8412	0.8411
	Bi-LSTM	0.84645	0.84545	0.8471	0.85485
	MSLSTM-CPO	0.85225	0.8522	0.85235	0.8529

---

 | Show Table

DownLoad: CSV

Figure 7.  The evaluation results of MSLSTM-CPO with three deep learning models.

DownLoad: Full-Size Img PowerPoint

4.2.3.   Discussion

Currently, sentiment analysis tasks are generally well developed in both machine learning and deep learning approaches. This experimental design compares the MSLSTM-CPO model proposed in this paper with traditional machine learning models and deep learning models respectively. Based on the data in Tables 1 and 2, it can be concluded that the comprehensive performance of the MSLSTM-CPO model has some validity and reliability compared to these models.

The reason why the MSLSTM-CPO model proposed in this paper can achieve better performance can be explained from the perspectives of both machine learning models and deep learning models respectively. First, the focus of the sentiment analysis task is on the exploitation of textual content. Traditional machine learning methods focus on feature extraction from a large amount of labelled data, which leads to classification results. This approach neglects the contextual coherence of the text content and leads to poor model training results. Secondly, RNN and LSTM models with memory capability are commonly used in sentiment analysis tasks. However, only LSTM can adapt to data with long sequences for memorisation. Therefore, the MSLSTM-CPO model proposed in this paper is based on the LSTM. While a normal LSTM model construction is unidirectional and has only one layer, MSLSTM-CPO is a bidirectional and multilayer model. As a result, the model is enhanced in terms of the number of layers, achieving a certain degree of balance between depth and breadth.

In summary, the MSLSTM-CPO model proposed in this paper can effectively implement a new type of college opinion analysis. And compared with machine learning and deep learning baseline methods, MSLSTM-CPO has better classification performance.

5.   Conclusions

This paper successfully proposes a MSLSTM-CPO model based on multi-scale deep learning for sentiment analysis of university students. Firstly, the method performs a word embedding operation on the text. Secondly, Bi-LSTM is chosen as the basic networking unit for layer superposition, and the sentiment classification results are obtained by embedding layer, LSTM layer, average layer and linear output layer. The limitations of traditional methods are balanced by the combination of depth and breadth of the model. Experiments on real datasets show that the MSLSTM-CPO model exhibits better performance than traditional machine learning models and commonly used deep learning models.

In future work, it is expected that the user's image and audio information can be mined and fused with textual information in the future to better improve the efficiency and accuracy of sentiment analysis of campus opinion. In addition, this study will further explore more machine learning and deep learning methods in future work. Their performance characteristics in real-world applications will be analysed in order to investigate sentiment analysis tasks based on deep learning frameworks more effectively. This will help to promote a positive attitude towards life and the development of mental health among university students.

6.   Acknowledgement

This work was funded by the Researchers Supporting Project number (RSPD2023R681) King Saud University, Riyadh, Saudi Arabia, and also funded by Humanities and Social Sciences Project of Chongqing Municipal Education Commission (22SKSZ047).

Conflict of interest

The authors declare there is no conflicts of interest.