Self voting classification model for online meeting app review sentiment analysis and topic modeling

Dataset description

The dataset is extracted from the Google Play store for several online meeting apps including ‘Google Meet’, ‘Goto Meeting’, ‘Zoom Meeting’, ‘Skype’, ‘Hangouts’, ‘Microsoft Teams’, and ‘Webex Meeting’. These apps have been selected regarding their overall rating on the Google Play store. The app’s reviews are extracted for the period of 12 October 2018 to 7 December 2021. This study considers only the reviews given in the English language. The reviews are collected using the Google Play scraper library. The collected dataset contains the review id, user name, the content of reviews, score by user for the app, thumbs up count, review created version, and data for the review posted. The reviews contain opinions of users regarding particular positive and negative features of an app. Besides criticism, such reviews also contain suggestions for improvement or the addition of new features. These reviews are very helpful for app companies to make changes according to user sentiments. Sample data from the collected dataset is shown in Table 1. The number of reviews varies for each app and the distribution of reviews is provided in Fig. 2.

Preprocessing steps

Preprocessing is an important part of text analysis which helps to reduce the complexity of feature vectors and improves models’ performance (Mehmood et al., 2017). The extracted dataset contains irrelevant and redundant information which can be removed to reduce the feature complexity without affecting the models’ performance. Several preprocessing steps are used to clean data such as removal of numbers, removal of punctuation, conversion to lowercase, stemming, and removal of stopwords.

• Removal of numbers: Occasionally user reviews contain numbers that do not contribute to sentiment classification. These numbers are removed using the Python function isalpha() which ensures that only characters are forwarded for further preprocessing.

• Removal of punctuation: Text contain lots of punctuation marks that help humans understand the intended meaning. However, punctuation is not useful for sentiment analysis using machine learning models. The punctuation marks are removed to reduce feature complexity.

• Convert to lowercase: This preprocessing step helps to reduce the complexity of the feature vector. Feature extraction techniques consider lower and upper case words as unique words. For example, ’User’, ’user’, and ’USER’ convey the same meaning for humans but feature extraction techniques treat them as unique words. Conversion to lowercase helps to reduce complexity.

• Stemming: Stemming is another very helpful preprocessing step to reduce the feature complexity. It changes different forms of the same word to its root form. For example, ’go’, ’going’ and ’goes’ are changed to their basic form ’go’. Porter stemmer library is used for this purpose.

• Removal of stopwords: Text contain lots of stopwords to improve text readability for humans, for machine learning approaches, they are useless. Consequently, removing the words such as ’is’, ’an’, ’the’, ’and’ etc. helps to reduce the feature set and improve classification performance.

Sample text data from the collected dataset, before and after the preprocessing steps is shown in Table 2.

Valence aware dictionary for sentiment reasoning

VADER is used for sentiment extraction from text data. VADER analyzes the polarity and sensitivity of sentiment in the text and finds the sentiment score by adding the intensities of each word in the text (Hutto & Gilbert, 2014). The sentiment score range varies between −4.0 to +4.0, where −4 is the most negative and +4 is the most positive sentiment score. The midpoint 0 represents a neutral sentiment. Figure 3 shows the ratio of positive, negative, and neutral sentiments in the dataset extracted using VADER.

Latent dirichlet allocation

LDA is a modeling technique used to extract topics from a text corpus. Latent means ‘hidden’ which shows that it is used to extract hidden topics in data (Blei, Ng & Jordan, 2003). LDA is based on Dirichlet distributions and processes and uses two metrics for topic modeling. Probability distribution of topics in documents and probability distribution of words in topics are used for topic modeling (LDA, 2018).

Feature engineering

The feature extraction techniques are required for training the machine learning models. This study uses three feature extraction techniques to train the models.

Bag of words

The bag of words (BoW) is the simplest technique used for feature extraction from text data (Rustam et al., 2021a). The BoW technique counts the appearance of each unique term from the corpus and makes a vector for the machine learning models. Depending upon the number of occurrences of different words, text similarity can be determined using the BoW feature vector. BoW features are extracted using the CountVectorizer Sci-Kit learn library.

Table 2:

Preprocessing results on sample reviews.

Reviews	After preprocessing
I would prefer to see the app show any video calls in a minimized window on movile devices like it would in the past.	prefer see app show any video call minimizi window movile devic past
I think they’re actively trying to make it worse.	think activi try make worse

DOI: 10.7717/peerjcs.1141/table-2

Machine learning models

This study uses four machine learning models including SVM, DT, LR, and KNN to validate the proposed self-voting approach. These models are used with their best hyperparameters setting according to the dataset. To select the best hyperparameters values ranges are obtained from the literature and fine-tuned to obtain the best performance (Rupapara et al., 2021a; Mujahid et al., 2021). The hyperparameter setting and tuning range are given in Table 3.

Self-Voting classifier

This study proposes a novel voting classifier, called a self-voting classifier. Traditional ensemble models follow a group voting mechanism, using heterogeneous models where the output of multiple models is combined using soft or hard voting criteria. Since the performance of different models varies, combining the prediction of multiple models improves the classification performance (Rustam et al., 2020b; Rustam et al., 2019; Rupapara et al., 2021b). Contrary to the group voting from heterogeneous models, this study adopts the self-voting ensemble where the output of the three different variants of SVM is combined to make the final prediction. Since the performance of a model varies concerning the features fed for training, the idea is to feed multiple features to the same model and combine them to make the ensemble. Three SVM variants have been trained on different feature vectors including BoW, TF-IDF, and hashing features. The performance of the self-voting approach is investigated both using the soft and hard voting criteria.

Figure 4 shows the process followed for soft voting (SV) where the probabilities predicted from each SVM variant is considered to calculate the average prediction probability of each class. The SVM-SV approach follows these steps. First, TF-IDF features are used for training the SVM using Eq. (3). (3)${\displaystyle tfidf=t{f}_{p,q}\ast log\left(\frac{N_r}{D_q}\right)}$

where tfidf gives weights for terms in the corpus using the TF-IDF. (4)${\displaystyle tfid{f}_{set}=\left(\begin{array}{cccc}\hfill F_1\hfill & \hfill F_2\hfill & \hfill ...\hfill & \hfill F_m\hfill \\ \hfill \hfill \\ \hfill tfid{f}_{1x1}\hfill & \hfill tfid{f}_{1x2}\hfill & \hfill ...\hfill & \hfill tfid{f}_{1xm}\hfill \\ \hfill tfid{f}_{2x1}\hfill & \hfill tfid{f}_{2x2}\hfill & \hfill ...\hfill & \hfill tfid{f}_{2xm}\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill tfid{f}_{nx1}\hfill & \hfill tfid{f}_{nx2}\hfill & \hfill ...\hfill & \hfill tfid{f}_{nxm}\hfill \\ \hfill \hfill \end{array}\right).}$

The tfidf_set is a feature set extracted using the TF-IDF technique and m is the number of features.

The unique words that belong to (N_r) number of reviews can be represented as (5)${\displaystyle f_1,f_2,\dots ,f_nϵN_{r}and{N}_{=}n.}$

Similar to TF-IDF, two SVM variants are trained on BoW and hashing features, respectively. (6)${\displaystyle bow=Count\left(t,N_{r,i}\right)}$

where BoW is the count of term t in a review N_(r,i) where N_(r,i)ϵN_r and below bow_set is a feature set extracted using the BoW technique. (7)${\displaystyle bo{w}_{set}=\left(\begin{array}{cccc}\hfill F_1\hfill & \hfill F_2\hfill & \hfill ...\hfill & \hfill F_m\hfill \\ \hfill \hfill \\ \hfill bo{w}_{1x1}\hfill & \hfill bo{w}_{1x2}\hfill & \hfill ...\hfill & \hfill bo{w}_{1xm}\hfill \\ \hfill bo{w}_{2x1}\hfill & \hfill bo{w}_{2x2}\hfill & \hfill ...\hfill & \hfill bo{w}_{2xm}\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill bo{w}_{nx1}\hfill & \hfill bo{w}_{nx2}\hfill & \hfill ...\hfill & \hfill bo{w}_{nxm}\hfill \\ \hfill \hfill \end{array}\right)}$

For hashing features, the feature set can be defined as (8)${\displaystyle h=hash\left(str\right)=str\left[0\right]+str\left[1\right]p{n}^1+...+str\left[n\right]p{n}^n}$

where h is the value of a string (str) calculated using hashing vectorizer function, pn is a prime number, str[i] is a character code, q is the index value and p is the value for the number of str strings. (9)${\displaystyle has{h}_{set}=\left(\begin{array}{cccc}\hfill F_1\hfill & \hfill F_2\hfill & \hfill ...\hfill & \hfill F_m\hfill \\ \hfill \hfill \\ \hfill h_{1x1}\hfill & \hfill h_{1x2}\hfill & \hfill ...\hfill & \hfill h_{1xm}\hfill \\ \hfill h_{2x1}\hfill & \hfill h_{2x2}\hfill & \hfill ...\hfill & \hfill h_{2xm}\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill .\hfill & \hfill .\hfill & \hfill \hfill & \hfill .\hfill \\ \hfill h_{nx1}\hfill & \hfill h_{nx2}\hfill & \hfill ...\hfill & \hfill h_{nxm}\hfill \\ \hfill \hfill \end{array}\right).}$

Using the tfidf_set, bow_set, and hash_set feature sets, three SVM variants are trained as follows (10)${\displaystyle sv{m}_{t1}=SVM\left(tfid{f}_{set}\right)}$ (11)${\displaystyle sv{m}_{t2}=SVM\left(bo{w}_{set}\right)}$ (12)${\displaystyle sv{m}_{t3}=SVM\left(has{h}_{set}\right)}$

where svm_t1, svm_t2, and svm_t3 are trained SVM using each feature set and can be combined to make the final prediction using SV criteria. (13)${\displaystyle po{s}_{p1},ne{g}_{p1},ne{u}_{p1}=sv{m}_{t1}\left(T{D}_{features}\right)}$ (14)${\displaystyle po{s}_{p2},ne{g}_{p2},ne{u}_{p2}=sv{m}_{t2}\left(T{D}_{features}\right)}$ (15)${\displaystyle po{s}_{p3},ne{g}_{p3},ne{u}_{p3}=sv{m}_{t3}\left(T{D}_{features}\right)}$

where pos_p, neg_p, and neu_p are probabilities for positive, negative, and neutral target classes, respectively and TD_features are features for test samples. (16)${\displaystyle p1=\frac{po{s}_{p1}+po{s}_{p1}+po{s}_{p1}}{3}}$ (17)${\displaystyle p2=\frac{po{s}_{p2}+po{s}_{p2}+po{s}_{p2}}{3}}$ (18)${\displaystyle p3=\frac{po{s}_{p3}+po{s}_{p3}+po{s}_{p3}}{3}}$

where p1, p2, and p3 are probabilities for positive, negative, and neutral classes using TF-IDF, BoW, and Hashing features, respectively. SVM-SV uses the argmax function in the end to find the class with the highest probability. (19)${\displaystyle finalprediction=argmax\left\{p1,p2,p3\right\}.}$

For hard voting (HV), the predicted class from each SVM variant is considered for the final prediction, as shown in Fig. 5. SVM-HV method uses majority voting criteria to make the final prediction. Each SVM variant predicts a target class (positive, negative, or neutral) using each feature set and then the SVM-HV performs voting on the predicted class. In case of a tie in voting, a higher weight is awarded to the minority class in the dataset which is the neutral class for this dataset. (20)${\displaystyle p1=SVM\left(tfid{f}_{set}\right)}$ (21)${\displaystyle p2=SVM\left(bo{w}_{set}\right)}$ (22)${\displaystyle p3=SVM\left(has{h}_{set}\right)}$

where p1, p2, and p3 are predictions by SVM variants with different feature sets. The majority voting function is used on these predictions to make the final prediction. In the case of tie final predictionϵminority class. (23)${\displaystyle finalprediction=mode\left\{p1,p2,p3\right\}.}$

 
 
 
Algorithm 1 Proposed SVM-SV algorithm 
 
    Input: Apps Reviews 
    Output: Positive|Negative|Neutral 
 1:  Def Model_Training(): 
  2:      SV MT ← SVM(TF-IDF_Features) 
  3:      SV MB ← SVM(BoW_Features) 
  4:      SV MH ← SVM(Hashing_Features) 
  5:  for i in Test_Corpus do 
 6:     P1 ← SV MT(i) 
  7:     P2 ← SV MB(i) 
  8:     P3 ← SV MH(i) 
  9:     SV M − SV (Pred) ← mode{P1,P2,P3} 
10:  end for 
11:  Positive|Negative|Neutral ← SV M − SV prediction 
    

Algorithm 1 shows the steps of the proposed SVM-SV model. Three different variants of SVM are trained as shown in lines 2 to 4 of Algorithm 1, where SVM_T indicates the SVM model trained using TF-IDF features, SVM_B is the model trained with BoW features while SVM_H is the model trained using Hashing features. Lines 6 to 8 show the predictions made from each model where P1, P2, and P3 are the prediction by trained SVM_T, SVM_B and SVM_H, respectively. In the end, mode of P1, P2, and P3 are taken to predict the final label of the test sample. It is the soft voting criterion; it infers that if two models out of three predict the test sample as positive, the final prediction will be positive.

Algorithm 2 shows the steps of the proposed SVM-HV model. Its input is app reviews while the output is the label of the sentiments for particular reviews. Three SVM are trained using each feature extraction method where SVM_T, SVM_B, and SVM_H indicate the models trained using TF-IDF, BoW, and Hashing features, respectively. In this algorithm, the probability of each sentiment is taken from each trained model, where Pos_i, Neg_i, and Neu_i are probabilities by each model for positive, negative, and neutral target classes, respectively, as given in lines 6 to 14 of Algorithm 2. Prob_Pos, Prob_Neg, and Prob_Neu are the average probabilities that are calculated using all models’ probabilities. Average probability is calculated by taking the summation of the probability of positive class from each model and dividing it by 3. The same procedure is adopted for negative and neutral sentiments. In the end, the argmax is taken to predict the final label for the test sample.

 
 
Algorithm 2 Proposed SVM-HV algorithm 
 
    Input: Apps Reviews 
    Output: Positive|Negative|Neutral 
 1:  Def Model_Training(): 
  2:      SV MT ← SVM(TF-IDF_Features) 
  3:      SV MB ← SVM(BoW_Features) 
  4:      SV MH ← SVM(Hashing_Features) 
  5:  for i in Test_Corpus do 
 6:     Pos1 ← SV MT(i) 
  7:     Neg1 ← SV MT(i) 
  8:     Neu1 ← SV MT(i) 
  9:     Pos2 ← SV MB(i) 
10:     Neg2 ← SV MB(i) 
11:     Neu2 ← SV MB(i) 
12:     Pos3 ← SV MH(i) 
13:     Neg3 ← SV MH(i) 
14:     Neu3 ← SV MH(i) 
15:     Prob_Pos ← (Pos1+Pos2+Pos3) 
             3 
16:     Prob_Neg ← (Neg1+Neg2+Neg3) 
             3 
17:     Prob_Neu ← (Neu1+Neu2+Neu3) 
              3 
18:     SV M − HV (Pred) ← argmax{Prob_Pos,Prob_Neg,Prob_Neu} 
19:  end for 
20:  Positive|Negative|Neutral ← SV M − HV prediction 
    

Experimental setup

For experiments, this study used an Intel Core i7 11th generation machine with the Windows operating system. To implement the proposed approach, Jupyter notebook is used with the Python language and Sci-kit learn, TensorFlow, NLTK, and Pandas libraries are used. Data splitting is done for model training and testing in ratios of 80% and 20%, respectively. The dataset contains three target classes including positive, negative, and neutral. The number of samples in the dataset after the data split is given in Table 4.

Results for sentiment classification

Table 5 shows the results of SVM with BoW, TF-IDF, and hashing features. It also contains the results of proposed approaches SVC-SV and SVC-HV. SVM performs significantly better with TF-IDF and hashing features and obtained a 0.98 accuracy score with each approach. On the other hand, BoW features do not show good results and SVM has a 0.95 accuracy score. The performance with TF-IDF and hashing features is more significant because of the significant feature sets generated by these techniques. TF-IDF assigns weight to each feature and shows better results as compared to simple term count from the BoW technique. Similarly, hashing generates a less complex feature set for model training which helps to increase models’ performance. SVC-SV is also good, similar to other features with SVM, however, SVC under hard voting under majority voting criteria outperforms all other approaches with a 1.00 accuracy score. This significant performance is primarily based on the combination of multiple variants of SVM trained on different features. It can be observed that different SVM variants show different per class accuracy for positive, negative, and neutral classes. For example, SVM with TF-IDF is good for the neutral class while using hashing feature is good to obtain the best performance for the positive class. Combining these variants trained on different features helps to obtain the best performance in all the classes as the SVM variants complement each other.

The self-voting approach has been validated using several machine learning models including DT, KNN, and LR. Table 6 shows the results using the DT model in terms of accuracy, precision, recall, and F1 score. Other than the self-voting approach, DT shows the best result when used with BoW features and obtains a 0.87 accuracy score as compared to TF-IDF and hashing features. DT is a simple rule-based model and can perform better using a simple feature set such as extracted by the BoW. DT with TF-IDF and hashing has marginally low performance with a 0.86 accuracy score for each feature set. The best performance is obtained when it is used with SVC-HV with a 0.88 accuracy score. Besides accuracy, precision, recall, and F1 score values are also superior to that of other features.

Table 7 shows the performance results of the LR model using BoW, TF-IDF, hashing features, and the SVC approach. LR shows better performance as compared to DT, however, its performance is inferior to SVM. LR performance with the SVC approach is more significant as compared to an individual feature but SVC-SV achieved a 0.95 accuracy score which is the highest as compared to results using other features.

KNN is another model that is used for experiments deployed with the proposed SVC approach. Experimental results given in Table 8 indicate that the proposed approach shows significant improvements over other approaches. On average, the performance of KNN is not good as compared to SVM, DT, and LR as it has accuracy scores of 0.75, 0.76, and 0.76 when used with BoW, TF-IDF, and hashing features, respectively. KNN tends to show poor performance with large datasets as compared to linear models such as SVM and LR which are more suitable for large feature sets, such as the dataset used in this study. Using the proposed SVC approach, the accuracy score of KNN is improved to 0.78 from 0.76.

Performance of deep learning models on apps reviews dataset

In comparison with our proposed approach using the machine learning models, this study also deploys some state of the arts deep learning models. For this purpose, long short-term memory (LSTM) (Rupapara et al., 2021a), gated recurrent unit(GRU) (Dey & Salem, 2017), convolutional neural networks(CNN) (Luan & Lin, 2019), and recurrent neural networks (RNN) are used. The architecture of these models is presented in Table 9.

The models use dropout layers, dense layers, and embedding layers as common among all models. The dropout layer is used to reduce the probability of model over-fitting and reduces the complexity of model learning by dropping neurons randomly. The embedding layer takes input and converts each word in reviews into vector form for model training. The dense layer is used with three neurons and a Softmax activation function to generate the desired output. Models are compiled with categorical cross-entropy function because of multi-class data and ‘adam’ optimizer is used for parameters optimization (Zhang, 2018). In the end, all models are fitted with 100 epochs and a batch size of 64.

Experimental results using deep learning models are given in Table 10. Results show that LSTM and GRU outperform other deep learning models with 0.92 and 0.91 accuracy scores, respectively. The performance of LSTM and GRU shows that the recurrent architecture model shows significantly better performance than other models on text data. RNN is also better compared to CNN which has the lowest accuracy of 0.81. The mechanism of eliminating unused information and storing the sequence of information makes recurrent applications a strong tool for text classification tasks. On the other hand, CNN requires a large feature set to perform better which in the case of this study does not seem so.

Comparison with other studies

The performance of the proposed approach is compared with other recent studies on sentiment analysis. In this regard, the state-of-the-art models from previous studies are deployed on the current dataset and the results are compared. First, the study (Rustam et al., 2019) used an ensemble model which is the combination of LR and stochastic gradient descent classifier (SGDC) for sentiment classification. The ensemble model is deployed on the current dataset and it obtained a 0.90 accuracy score. The study (Rustam et al., 2021b) used a hybrid approach for sentiment classification related to COVID-19 tweets. The study used an extra tree classifier and feature union technique for sentiment classification. The study (Rustam et al., 2020a) used a hybrid approach which is a combination of TF-IDF features, Chi-square feature selection technique, and LR model. The study (Tam, Said & Tanriöver, 2021) proposed a hybrid model ConvBiLSTM using CNN and BiLSTM networks for tweets sentiment classification and similarly, another study (Jain, Saravanan & Pamula, 2021) proposed a hybrid model CNN-LSTM for sent for consumer sentiment analysis. Performance comparison results of these studies are provided in Table 11.

Statistically significant T-test

A statistical T-test is performed to show the significance of the proposed approach. T-test accepts the null hypothesis if the compared values are statistically the same and reject the null hypothesis if the compared values are statistically different (Omar et al., 2021). We deploy the T-test on the models’ performance with each feature and the proposed self-voting. We evaluate performance in terms of T-statistic and critical value (CV). The T-statistic value is greater than the CV in all cases which means that for all cases the null hypothesis is rejected. T-statistic results are shown in Table 12. These results show that all cases are statistically different in comparison with the proposed approach.

LDA topic extraction and topic sentiment visualization

This study also carried out topic modeling using the LDA approach. The topics are extracted from all app reviews, as well as, each app review to show the topic-wise users’ sentiments. We used the LDA model to extract the top four topics from review data.

For topic modeling, the LDA is used with three hyperparameters including n_components, random_state, and evaluate_every. The n_components parameter is used with value four indicating that four topics will be extracted with this setting; random_state with value is 10, and evaluate_every value is −1. The most commonly discussed topics are ‘easy use’, ‘join meeting’, ‘online class’, and ‘virtual background’. We illustrate these topic counts and sentiments for each topic in Fig. 6. It shows that the majority of the positive comments are posted for ease of use for the online meeting apps followed by the virtual background provided by these apps. Although the ratio of negative sentiments is approximately three times low as compared to positive sentiments, most of the negative sentiments are given to joining meetings and ease of use attributes.

The patterns of sentiments for different topic is almost similar for all the apps under discussion; the distribution of topics discussed may slightly vary. Similarly, the positive and negative words used for different apps may vary as well. For example, the negative words used for the Google Meet app are horrible, sad, weak, irritated, etc.

Sentiments for common topics discussed for the Zoom app indicate that the ratio of negative sentiments for topics is slightly less than in the Google Meet app. Similarly, the number of positive words is less comparatively and negative words are slightly different such as sorry, awful, terrible, etc.

Topic sentiments and negative and positive words used for the Goto meeting app indicate that the number of topic sentiments is substantially higher than in Zoom and Google Meet apps. The ratio of negative topic sentiments is also low than both Zoom and Google Meet apps. The pattern of negative word usage is almost similar to other apps.

Skype-related topic sentiments are very low as compared to other apps and the ratio of negative sentiments is substantially high. The patterns for positive and negative words are similar to other apps. For the Webex app, the number of sentiments is low as compared to other Zoom, and Google Meet apps, it shows a higher ratio of positive sentiments.

In the end, the topics-related sentiments and top words for the Microsoft team and Hangout apps are given. They have a low number of sentiments and a low ratio of negative sentiments for the discussed topics. Similarly, the used negative words are also slightly different than other apps like nasty, regret, and uncomfortable for Hangouts and atrocious, scary, and confusion for the Microsoft team app.

Existing studies report the superior performance of ensemble models over stand-alone machine learning models. So, this study adopts an ensemble approach for sentiment analysis of online meeting apps which have been prevalent recently, especially during the COVID-19 breakout. Traditional ensemble models merge heterogeneous models to get the best of them for obtaining higher performance. Contrary to this approach, this study makes an ensemble model out of a single model. Empirical findings show that the same model shows different performances concerning a feature vector used for training. So this study follows a feature-centric approach and different best-performing features are selected to train the same model. For this purpose, the SVM model is trained using TF-IDF, BoW, and hashing features for sentiment analysis. Experiments are performed using a large dataset of reviews for online meeting apps.

Results demonstrate that the self-voting model tends to improve the performance of stand-alone models. The performance of the models is enhanced regarding two important aspects. First, traditional ensembles use multiple models with a single feature vector for the most part. Although, the advantage of multiple models is obtained, the potential of multiple features is lost. Also, different models may not be suitable for the same data, and combing them may not be prudent. Secondly, it is more rational to use a single model with multiple features if it is performing well on data. Following this rationale, we utilized variants of a single model which are trained using different feature vectors and obtain superior performance. The performance of the self-voting models is much better than single models.