Analisis Sentimen Ulasan Aplikasi Mobile Legends Berbasis Pelabelan IndoBERT dan SMOTE dengan Komparasi Algoritma Klasifikasi Naive Bayes dan SVM

Abstract

This study was conducted to evaluate user sentiment trends toward the Mobile Legends: Bang Bang game app through reviews published on the Google Play Store platform. This research applied IndoBERT-based automatic labeling to generate sentiment labels that better represent the textual meaning of the reviews. The class distribution imbalance resulting from the labeling process was addressed using the Synthetic Minority Oversampling Technique (SMOTE) during the model training phase. The performance of two classification algorithms—the Naive Bayes Classifier and the Support Vector Machine (SVM)—was compared through hyperparameter tuning using GridSearchCV, with the F1-Macro metric. The results show that the Naive Bayes Classifier model delivers the best performance with an accuracy of 87.55%, an F1-Score of 85.78%, and an F1-Score of 56.70%. However, the model still exhibits significant limitations in recognizing the neutral sentiment class (F1-Score 0.21), which was further analyzed and found to be caused by the very small proportion of the neutral class (2.3% of the total data) as well as the characteristic of neutral reviews, which tend to be requests or suggestions to developers rather than explicit statements. This study contributes to the development of a more representative sentiment analysis methodology through a combination of transformer-based labeling, data imbalance handling, and classification algorithm comparison.