An ensemble approach for imbalanced multiclass malware classification using 1D-CNN

Phase-I: vectorizing Mal-API-2019

Dataset Description:

The Mal-API-2019 dataset consists of API sequence records of 7107 malwares of eight different classes. Figure 2 speaks about the frequency distribution of each class and Fig. 3 shows the frequency of malware samples considering the API sequence length before preprocessing.

Figures 2 and 3 shows that Mal-API-2019 is highly imbalanced. Such diverse distribution has adverse effect on the success of classification. The said dataset is interpreted as having two atrributes named API_seq and M_class. The former represents the sequence of APIs called by the malware and the latter represents the class of that malware. Each entry in API_seq is interpreted as a sentence of type M_class made up of finite number of words i.e., APIs with repetitions from the API_Vocabulary = {API₀, API₁, API₂, …., API_n}. The M_class ranges from 0 to 7 representing eight different type of senetences maping to 8 malware classes. For the representation of textual documents in a multidimensional vector space, the vector space model is particularly popular. In one of the previous work TF-IDF vectorization technique has been used on DLL sequences for host based anomaly detection (Panda & Tripathy, 2020). In a comparable manner, word embedding vector of each distinct API of API_Vocabulary is created using Skip-gram model of Word2Vec embedding technique. The redundant API occurrences are eliminated in order to address the dataset’s variability with regard to the length of each entry. The distribution of records in the processed dataset against record length following duplicate removal is shown in Fig. 4.

Finding Word Embeddings for each API:

The word embeddings are discovered using Skip-gram by taking into account all the records of API_seq for each unique API. Skip-gram does well with small datasets and depicts less common words better than more frequent ones (Mikolov et al., 2013). With the aim of capturing the embeddings (feature vectors) of each unique API that qualify the meaning of the API they represent, a skip-gram neural network model, as shown in Fig. 5, is utilized. Such feature vectors can be quite useful in describing the M_class type. For each target API, the skip-gram technique takes into account all windows of size ℓ in order to extract the semantics of the APIs into embeddings. The embedding matrix W₁ shown in Fig. 5, which is given to the 1D-CNN models for training purposes in Phase-II of the proposed architecture, serves as the word embeddings for all different APIs. The weight matrix W₂ can be employed to predict the likelihood of various words given a context word. Since the main goal of this effort is to obtain the word embeddings, W₂ is not utilized.

Phase-II: training 1D-CNN models

Due to their effectiveness, deep learning models are becoming more and more popular. The convolution layer, pooling layer, and fully connected artificial neural network layer (Dense Layer) are the three basic layers that make up the one-dimensional convolutional neural network (1D-CNN), a deep learning model. These models use convolution and pooling operations to learn features from sequential input, such as texts, and then conduct binary or multiclass classification in the dense layer. It performs the convolution operation with various kernels on the spatial input texts in the convolution layer to produce corresponding one-dimensional feature maps. (1)${\displaystyle x_j^l=f\left(\right.\sum _{i=1}^M{x}_i^{l-1}\ast k_{ij}^l+b_j^l\left)\right.}$ The operation of the one-dimensional convolution layer is as described in Eq. (1), where k and j represents convolution kernels and number of kernels respectively. M denotes channel number in input x^l−1 with b as the bias to the corresponding kernel. The ∗ is the convolution operator and f() is the activation function. The pooling layer uses the avarage pooling or max pooling method with a predetermined window to reduce the feature dimension produced by the convolution operation. Output of the final pooling layer l + 1 is given as input to the dense layer and the output of the dense layer is evaluted as described in Eq. (2), where w and b denotes weight and bias respectively. (2)${\displaystyle h\left(x\right)=f\left(\right.w^{l+1}.x^{l+1}+b^{l+1}\left)\right.}$ The architecture and description of the 1D-CNN model utilised in this study for training and testing are shown in Fig. 6. Following the procedures outlined in Algorithm-1, each 1D-CNN classifier identified as model_c in Fig. 1 is trained using Mal_API_c an intermediate dataset of the processed dataset. The statements 2-13 in Algorithm-1 generates the set OvR_Mal_API_Datasets, which is a collection of intermediate datasets. Every intermediate dataset present in OvR_Mal_API_Datasets labels the record containing the “class of interest” as 1 (positive) and the ”rest all” as 0 (negative). The statements 14-20 trains 1D-CNN classifiers corresponding to each of the intermediate dataset present in OvR_Mal_API_Datasets and returns 1D_CNN_Classifier_List, a list of classifiers. The third phase of the architecture combines the capabilities of all of these trained models.

Phase-III: ensemble with SoftVoting

Using the ModifiedSoftVoting method outlined in Algorithm-2, the classification abilities of independently trained classifiers in the 1D_CNN_Classifier_List are ensembled. In this phase a Test Set is created from the processed dataset using stratified sampling as shown in statement 2 of the algorithm. Based on the outcomes of all the trained classifiers for each record in the Test Set, this algorithm determines the best predicted class. In the 1D_CNN_Classifier_List, each classifier (model_c) predicts two probability scores of each record in the Xtest as ‘not in class of interest’ and ‘in class of interest’ respectively as per statements 3-7 of the algorithm. ClassificationScore( c_score) is a two dimensional array with a size of (mX2) where m represents count of records in Xtest . The i^th row of c_score represents probability score for the i^th record of Xtest as ‘not in class of interest’ and ‘in class of interest’ at index 0 and 1 respectively. ClassProbability( c_proba) is a two dimensional array of size (mX8) where m represents count of records in Xtest. The j^th column of c_proba will contain m probability scores of m records of Xtest as ‘in class of interest’ using j^th classifier model_j in 1D_CNN_Classifier_List. The i^th row of c_proba represents probability scores of i^th record of Xtest as ‘in class of interest’ for all classifiers in 1D_CNN_Classifier_List respectively.

Statements 8-13 in Algorithm-2 constructs the confusion matrix(cm) by ensembling all the predictions. The cm is a square matrix of dimention (nXn) where n is the count of distinct classes in M_class. It contributes to the estimation of several performance metrics as mentioned in Eqs. (3), (4), (5) and (6). The fundamental parameters required to calculate various performance metrics are TP , FN, FP, and TN. These parameters for a specific class of interest is interpreted in the cm as a case of:

1. TP (True Positive): When malware of “class of interest” is predicted as “class of interest”

2. FN (False Negative): When malware of “class of interest” is predicted as some other class.

3. FP (False Positive): When some other class of malware is predicted as malware of “class of interest”

4. TN (True Negative): When malware of other class is predicted as malware of other class.

Tables 1 and 2 represents two exemplary cases of confusion matrix and TP, FP, FN, and TN parameters for specific “class of intertest” C₃ and C₂ respectively in the context of multiclass problem.

Table 1:

When class of interest is Class-3.

DOI: 10.7717/peerjcs.1677/table-1

Table 2:

When class of interest is Class-2.

DOI: 10.7717/peerjcs.1677/table-2

Accuracy of the model is estimated as count of correct predictions divided by total count of predictions. Equation (3) mathematically represents the calculation of accuracy. Sometimes accuracy may mislead, hence the performance is ensured by calculating average of precision, recall and F₁ score respectively for all classes in multiclass classification problem. (3)${\displaystyle Accuracy=\frac{\sum _{i=1}^{n}cm\left[i,i\right]}{\sum _{i=1}^n\sum _{j=1}^{n}cm\left[i,j\right]}}$ (4)${\displaystyle Precisio{n}_c=\frac{cm\left[c,c\right]}{\sum _{i=1}^{n}cm\left[i,c\right]}}$ (5)${\displaystyle Recal{l}_c=\frac{cm\left[c,c\right]}{\sum _{i=1}^{n}cm\left[c,i\right]}}$ (6)${\displaystyle {F_1}_c=\frac{2}{\frac{1}{Recal{l}_c}+\frac{1}{Precisio{n}_c}}}$ Precision for a specific class c (Precision_c) is estimated to see the impact of FP as higher concern than FN, as explained in Eq. (4). It is estimated as the number of true positives devided by the number of predicted positives. Recall for a specific class c (Recall_c) is estimated to see the impact of FN as higher concern than FP. It is estimated as the number of true positives divided by total number of actual positives. F₁-Score of a specific class c (F_1c) is the harmonic mean of Precision_c and Recall_c. It is used to ensure high precision against high recall. Weighted and macro average of precision, recall and F₁-score are used as perfomace metrics for multiclass problems. Unweighted mean of each of these performance metrics are referred as macro average measure. Weighted mean of each of these performance metrics are referred as weighted average measure using count of samples of each class as the weight.