Analisis Business Intelligence Ulasan Negatif Aplikasi M-Pajak Menggunakan BERTopic untuk Evaluasi Layanan Digital

Henry Pandia

doi:10.47065/mis.v4i3.2848

Henry Pandia * Universitas Advent Indonesia, Indonesia

DOI: https://doi.org/10.47065/mis.v4i3.2848

Keywords: BERTopic; Business Intelligence; App Store Review Mining; Layanan Perpajakan Digital; Analisis Ulasan Negatif; Topic Modeling; M-Pajak; Google Play Store

Abstract

The M-Pajak application is part of the government’s digital tax service transformation aimed at improving the efficiency and accessibility of tax services for the public. However, user complaints on the Google Play Store indicate that technical and operational problems persist with the application. This study aims to analyze negative user reviews of the M-Pajak application using BERTopic to generate Business Intelligence insights related to the quality of government digital tax services. BERTopic was selected because it uses transformer-based contextual embeddings, which are well-suited to analyzing short, unstructured, and informal mobile application reviews, thereby helping identify complaint themes in a more context-aware way than traditional word-distribution-based topic modeling approaches. The data were collected from Google Play Store reviews from May 2023 to May 2026, totaling 6,024 user reviews. A total of 5,008 negative reviews with ratings of 1 or 2 were used as the primary focus of the analysis. The research stages included data scraping, preprocessing, BERTopic modeling, topic evaluation, and Business Intelligence interpretation. The results show that the main user problems are related to authentication and login, system errors, email and OTP verification, NPWP and NIK registration, and digital service stability. Model evaluation yielded a Topic Coherence score of 0.522, a Topic Diversity score of 0.667, and a Topic Quality score of 0.348, indicating that the topic quality was moderate yet interpretable. The contribution of this study lies in utilizing negative reviews of a digital tax service application as a source of operational insights to support the evaluation and improvement of government digital public services.

References

R. Rachmiani, N. K. Oktadinna, and T. R. Fauzan, “The impact of online reviews and ratings on consumer purchasing decisions on e-commerce platforms,” International Journal of Management Science and Information Technology, vol. 4, no. 2, pp. 504–515, 2024, doi: https://doi.org/10.35870/ijmsit.v4i2.3373

A. I. Alfanzar, Khalid, and I. S. Rozas, “Topic Modelling Skripsi Menggunakan Metode Latent Diriclhet Allocation,” JSiI (Jurnal Sistem Informasi), vol. 7, no. 1, pp. 7–13, 2020, doi: https://doi.org/10.30656/jsii.v7i1.2036

M. Jelita, “Text Mining dengan Topic Modelling LDA dari Pertanyaan Gelar Wicara Literasi Perpustakaan Nasional RI,” Media Pustakawan, vol. 31, no. 3, pp. 253–265, 2024, doi: https://doi.org/10.37014/medpus.v31i3.5237

A. Syaifuddin, R. A. Harianto, and J. Santoso, “Analisis trending topik untuk percakapan media sosial dengan menggunakan topic modelling berbasis algoritme LDA,” Journal of Intelligent System and Computation, vol. 2, no. 1, pp. 12–19, 2021, doi: https://doi.org/10.52985/insyst.v2i1.150

F. Bianchi, S. Terragni, and D. Hovy, “Pre-training is a hot topic: Contextualized document embeddings improve topic coherence,” arXiv preprint arXiv:2004.03974, 2021. doi: https://doi.org/10.48550/arXiv.2004.03974

M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv preprint arXiv:2203.05794, 2022. doi: https://doi.org/10.48550/arXiv.2203.05794

R. Kandala, N. Vanhasbroeck, and K. Hoemann, “FLAME: A new dataset on FLemish accounts of momentary experiences,” arXiv preprint arXiv:2504.14707, 2026. doi: https://doi.org/10.48550/arXiv.2504.14707

R. Karthick, “Context-aware topic modeling and intelligent text extraction using transformer-based architectures,” Journal of Science Technology and Research (JSTAR), vol. 6, no. 1, pp. 1–12, 2025, doi: https://doi.org/10.2139/ssrn.5275391

A. Shroff, B. J. Shah, and H. Gajjar, “Online food delivery research: A systematic literature review,” International Journal of Contemporary Hospitality Management, vol. 34, no. 8, pp. 2852–2883, 2022, doi: https://doi.org/10.1108/IJCHM-10-2021-1273

W. Martin, F. Sarro, Y. Jia, Y. Zhang, and M. Harman, “A survey of app store analysis for software engineering,” IEEE Transactions on Software Engineering, vol. 43, no. 9, pp. 817–847, 2017, doi: https://doi.org/10.1109/TSE.2016.2630689.

D. Rifaldi, A. Fadlil, and Herman, “Teknik preprocessing pada text mining menggunakan data tweet ‘Mental Health’,” DECODE: Jurnal Pendidikan Teknologi Informasi, vol. 3, no. 2, pp. 161–171, 2023. doi: https://doi.org/10.51454/decode.v3i2.131

S. Terragni, E. Fersini, B. G. Galuzzi, P. Tropeano, and A. Candelieri, “OCTIS: Comparing and optimizing topic models is simple!,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, Online, pp. 263–270, 2021, doi: https://doi.org/10.18653/v1/2021.eacl-demos.31

A. Upadhye, “Enhancing semantic understanding by visualizing sentence-level embeddings,” International Journal of Computer Applications, vol. 185, no. 46, pp. 20–24, 2023, [Online]. Available:

https://www.ijcaonline.org/archives/volume185/number46/33000-2023923275/

L. McInnes, J. Healy, and J. Melville, “UMAP: Uniform manifold approximation and projection for dimension reduction,” arXiv:1802.03426 [stat.ML], 2020, doi: https://doi.org/10.48550/arXiv.1802.03426

D. Vijayan and I. Aziz, “Adaptive hierarchical density-based spatial clustering algorithm for streaming applications,” Telecom, vol. 4, no. 1, pp. 1–14, 2023, doi: https://doi.org/10.3390/telecom4010001.

D. N. Amalina and A. Fauzan, “A Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) Approach for Identifying Potential Villages in Buleleng Regency,” Knowledge Engineering and Data Science, vol. 7, no. 2, pp. 187–199, 2024, doi: https://doi.org/10.17977/um018v7i22024p187-199

A. B. Dieng, F. J. R. Ruiz, and D. M. Blei, “Topic modeling in embedding spaces,” arXiv preprint arXiv:1907.04907, 2019. doi: https://doi.org/10.48550/arXiv.1907.04907

O. Babalola, B. Ojokoh, and O. Boyinbode, “Comprehensive evaluation of LDA, NMF, and BERTopic’s performance on news headline topic modeling,” Journal of Computing Theories and Applications, vol. 2, no. 2, pp. 268–289, 2024. [Online]. Available: publikasi.dinus.ac.id/index.php/jcta/article/view/11635

D. Karabašević, A. Vujko, V. Mirčetić, G. Popović, and D. Stanujkić, “A transformer-based semantic encoding framework for quantitative analysis of large-scale textual reviews,” Axioms, vol. 15, no. 3, p. 175, 2026, doi: https://doi.org/10.3390/axioms15030175

M. Röder, A. Both, and A. Hinneburg, “Exploring the space of topic coherence measures,” in Proceedings of the 8th ACM International Conference on Web Search and Data Mining (WSDM '15), Shanghai, China, pp. 399–408, 2015, doi: https://doi.org/10.1145/2684822.2685324

I. K. Mensah, “Impact of government capacity and e-government performance on the adoption of e-government services,” International Journal of Public Administration, vol. 43, no. 4, pp. 303–311, 2020, doi: https://doi.org/10.1080/01900692.2019.1628059

Rijal, Jusman, Nasrullah, and Sukmawati, “Efektivitas implementasi aplikasi layanan publik berbasis digital (Makassar Smart City) dalam meningkatkan kepuasan masyarakat,” Moderat: Jurnal Ilmiah Ilmu Pemerintahan, vol. 11, no. 4, pp. 1352–1375, 2025, doi: https://doi.org/10.25157/moderat.v11i4.5223