User Satisfaction with Chinese Government Apps: Topic Mining and Sentiment Analysis of User Reviews
DOI:
https://doi.org/10.52152/23.3.95-124(2025)Keywords:
mobile government services, online comments, topic modeling, sentiment analysis, satisfaction evaluationAbstract
This study investigates user satisfaction with Chinese government apps, employing topic mining and sentiment analysis techniques to examine a dataset of 32,660 reviews from 10 province-level government apps. Using the Latent Dirichlet Allocation (LDA) topic model, we developed a customized framework for evaluating user satisfaction. We conducted a comparative analysis of eight machine learning (ML) models to identify the most effective approach for sentiment analysis and satisfaction quantification. The findings reveal that user satisfaction with these apps is moderately low, indicating challenges in mobile government service quality. Notably, significant regional disparities were observed, with higher satisfaction in central and eastern regions compared to northeastern and western regions. This reflects the influence of socioeconomic, geographic, and cultural factors on digital governance outcomes. Based on these insights, actionable recommendations for improving government app design and enhancing mobile government service quality are proposed.
References
Aamir, T., Chhetri, M. B., Chamikara, M. A. P., & Grobler, M. (2023, September). Government mobile apps: Analysing citizen feedback via app reviews. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) (pp. 1858-1863). Piscataway, NJ: IEEE.
Ahn, H., & Park, E. (2023). Motivations for user satisfaction of mobile fitness applications: An analysis of user experience based on online review comments. Humanities & Social Sciences Communications, 10(1). doi:10.1057/s41599-022-01452-6
Airlangga, M. C., Sulasikin, A., Nugraha, Y., Husna, N. L. R., Aminanto, M. E., Kurniawan, F., & Kanggrawan, J. I. (2023, September). Understanding citizen feedback of Jakarta government super app: Leveraging deep learning models. In 2023 IEEE International Smart Cities Conference (ISC2) (pp. 1-6). Piscataway, NJ: IEEE.
Al-Hawari, A., Najadat, H., & Shatnawi, R. (2021). Classification of application reviews into software maintenance tasks using data mining techniques. Software Quality Journal, 29(3), 667-703. doi:10.1007/s11219-020-09529-8
Ali, H., & Al Kabbi, R. (2018). M-government applications: Measurement of users' satisfaction in the Kingdom of Bahrain. Electronic Government, 14(4), 375-388.
Araujo, A., Golo, M., Viana, B., Sanches, F., Romero, R., & Marcacini, R. (2020, October). From bag-of-words to pre-trained neural language models: Improving automatic classification of app reviews for requirements engineering. In Encontro Nacional de Inteligência Artificial e Computacional (ENIAC) (pp. 378-389). Porto Alegre, Brazil: Sociedade Brasileira de Computação.
Asnawi, M. H., Pravitasari, A. A., Herawan, T., & Hendrawati, T. (2023). The combination of contextualized topic model and MPNet for user feedback topic modeling. IEEE Access, 11, 130272-130286. doi:10.1109/access.2023.3332644
Bastani, K., Namavari, H., & Shaffer, J. (2019). Latent Dirichlet allocation (LDA) for topic modeling of the CFPB consumer complaints. Expert Systems with Applications, 127, 256-271. doi:10.1016/j.eswa.2019.03.001
Blair, S. J., Bi, Y. X., & Mulvenna, M. D. (2020). Aggregated topic models for increasing social media topic coherence. Applied Intelligence, 50(1), 138-156. doi:10.1007/s10489-019-01438-z
Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77-84.
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3(4-5), 993-1022. doi:10.1162/jmlr.2003.3.4-5.993
Boon-Itt, S., & Skunkan, Y. (2020). Public perception of the COVID-19 pandemic on Twitter: Sentiment analysis and topic modeling study. JMIR Public Health and Surveillance, 6(4), e21978.
Castilla, R., Pacheco, A., & Franco, J. (2023). Digital government: Mobile applications and their impact on access to public information. Softwarex, 22. doi:10.1016/j.softx.2023.101382
Chen, A. J., Pan, S. L., Zhang, J. L., Huang, W. W., & Zhu, S. M. (2009). Managing e-government implementation in China: A process perspective. Information & Management, 46(4), 203-212. doi:10.1016/j.im.2009.02.002
Chen, N., Su, X. D., Liu, T. Y., Hao, Q. Z., & Wei, M. (2020). A benchmark dataset and case study for Chinese medical question intent classification. Bmc Medical Informatics and Decision Making, 20. doi:10.1186/s12911-020-1122-3
Chen, T. G., Zhang, C., Yang, J. J., & Cong, G. D. (2022). Grounded theory-based user needs mining and its impact on app downloads: Exampled with WeChat app. Frontiers in Psychology, 13. doi:10.3389/fpsyg.2022.875310
Chetty, K., Liu, Q. G., Gcora, N., Josie, J., Li, W. W., & Fang, C. (2018). Bridging the digital divide: Measuring digital literacy. Economics: The Open Access, Open-Assessment E-Journal, 12. doi:10.5018/economics-ejournal.ja.2018-23
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13(3), 319-340. doi:10.2307/249008
de Lima, V. M. A., de Araújo, A. F., & Marcacini, R. M. (2022). Temporal dynamics of requirements engineering from mobile app reviews. PeerJ Computer Science, 7. doi:10.7717/peerj-cs.874
Dhar, S., & Bose, I. (2022). Walking on air or hopping mad? Understanding the impact of emotions, sentiments and reactions on ratings in online customer reviews of mobile apps. Decision Support Systems, 162. doi:10.1016/j.dss.2022.113769
Di Sorbo, A., Grano, G., Visaggio, C. A., & Panichella, S. (2021). Investigating the criticality of user-reported issues through their relations with app rating. Journal of Software-Evolution and Process, 33(3). doi:10.1002/smr.2316
Eid, R., Selim, H., & El-kassrawy, Y. (2021). Understanding citizen intention to use m-government services: An empirical study in the UAE. Transforming Government: People, Process and Policy, 15(4), 463-482. doi:10.1108/tg-10-2019-0100
Ekman, P. (1992). Are there basic emotions. Psychological Review, 99(3), 550-553.
Fang, F., Zhou, Y., Ying, S., & Li, Z. J. (2023). A study of the Ping An health app based on user reviews with sentiment analysis. International Journal of Environmental Research and Public Health, 20(2). doi:10.3390/ijerph20021591
Gu, E., Meng, T. G., Wang, H. Y., & Zhang, A. (2023). E-government use, perceived transparency, public knowledge of government performance, and satisfaction with government: An analysis of mediating, moderating, and framing mechanisms based on the COVID-19 outbreak control survey data from China. Social Indicators Research, 169(1-2), 79-124. doi:10.1007/s11205-023-03135-4
Gürler, S., Cavusoglu, B., Ozdamli, F., Mousa, K. M., & Baykan, H. (2024). Mobile government use and crisis management: The moderating role of techno-skepticism. Sustainability, 16(12). doi:10.3390/su16124904
Haggag, O., Grundy, J., Abdelrazek, M., & Haggag, S. (2022). A large scale analysis of mHealth app user reviews. Empirical Software Engineering, 27(7). doi:10.1007/s10664-022-10222-6
He, L., Jia, Y., Han, W. H., & Ding, Z. Y. (2014). Mining user interest in microblogs with a user-topic model. China Communications, 11(8), 131-144. doi:10.1109/cc.2014.6911095
Horan, T. A., Abhichandani, T., & Rayalu, R. (2006, January). Assessing user satisfaction of e-government services: Development and testing of quality-in-use satisfaction with advanced traveler information systems (ATIS). In Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06) (Vol. 4, p. 83b). Piscataway, NJ: IEEE.
Hu, H. Y., Bezemer, C. P., & Hassan, A. E. (2018). Studying the consistency of star ratings and the complaints in 1 & 2-star user reviews for top free cross-platform Android and iOS apps. Empirical Software Engineering, 23(6), 3442-3475. doi:10.1007/s10664-018-9604-y
Hu, T. Y., & Jiang, Y. (2019). Mining of user's comments reflecting usage feedback for app software. Journal of Software, 30(10), 3168-3185. doi:10.13328/j.cnki.jos.005794
Hung, S. Y., Chang, C. M., & Kuo, S. R. (2013). User acceptance of mobile e-government services: An empirical study. Government Information Quarterly, 30(1), 33-44. doi:10.1016/j.giq.2012.07.008
Jelodar, H., Wang, Y. L., Yuan, C., Feng, X., Jiang, X. H., Li, Y. C., & Zhao, L. (2019). Latent Dirichlet allocation (LDA) and topic modeling: Models, applications, a survey. Multimedia Tools and Applications, 78(11), 15169-15211. doi:10.1007/s11042-018-6894-4
Jiang, J. Y., Meng, T. G., & Zhang, Q. (2019). From Internet to social safety net: The policy consequences of online participation in China. Governance: An International Journal of Policy Administration and Institutions, 32(3), 531-546. doi:10.1111/gove.12391
Jou, Y. T., Mariñas, K. A., Saflor, C. S., Baleña, A., Gutierrez, C. J., Dela Fuente, G., . . . Young, M. N. (2024). Investigating various factors influencing the accessibility of digital government with eGov PH mobile application. Sustainability, 16(3). doi:10.3390/su16030992
Katz, E., Blumler, J. G., & Gurevitch, M. (1973). Uses and gratifications research. The Public Opinion Quarterly, 37(4), 509-523.
Kumar, A., Bala, P. K., Chakraborty, S., & Behera, R. K. (2024). Exploring antecedents impacting user satisfaction with voice assistant app: A text mining-based analysis on Alexa services. Journal of Retailing and Consumer Services, 76. doi:10.1016/j.jretconser.2023.103586
Larsson, K. K. (2021). Digitization or equality: When government automation covers some, but not all citizens. Government Information Quarterly, 38(1). doi:10.1016/j.giq.2020.101547
Li, Y., & Shang, H. P. (2020). Service quality, perceived value, and citizens' continuous-use intention regarding e-government: Empirical evidence from China. Information & Management, 57(3). doi:10.1016/j.im.2019.103197
Liu, Q., Liang, Y., Wang, S. Y., Huang, Z. G., Wang, Q., Jia, M. Y., . . . Ming, W. (2022). Health communication through Chinese media on E-cigarette: A topic modeling approach. International Journal of Environmental Research and Public Health, 19(13). doi:10.3390/ijerph19137591
Liu, X. P., & Zheng, L. (2018). Cross-departmental collaboration in one-stop service center for smart governance in China: Factors, strategies and effectiveness. Government Information Quarterly, 35(4), S54-S60. doi:10.1016/j.giq.2015.12.001
Malik, H., Shakshuki, E. M., & Yoo, W. S. (2020). Comparing mobile apps by identifying ‘Hot’features. Future Generation Computer Systems, 107, 659-669.
Mao, Z., Zou, Q., Bu, T., Dong, Y., & Yan, R. (2023). Understanding the role of service quality of government apps in continuance intention: An expectation-confirmation perspective. Sage Open, 13(4). doi:10.1177/21582440231201218
Miranda-Valencia, B. L. (2021). Satisfaction and consumption emotions of library users at a public university in Mexico: A case study. Libri-International Journal of Libraries and Information Studies, 71(2), 109-121. doi:10.1515/libri-2020-0002
Mizrahi, S., Cohen, N., & Vigoda-Gadot, E. (2020). Government's social responsibility, citizen satisfaction and trust. Policy and Politics, 48(3), 443-460. doi:10.1332/030557320x15837138439319
Ng, C. Y., Law, K. M. Y., & Ip, A. W. H. (2021). Assessing public opinions of products through sentiment analysis: Product satisfaction assessment by sentiment analysis. Journal of Organizational and End User Computing, 33(4), 125-141. doi:10.4018/JOEUC.20210701.oa6
Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17(4), 460-469.
Palomba, F., Linares-Vásquez, M., Bavota, G., Oliveto, R., Di Penta, M., Poshyvanyk, D., & De Lucia, A. (2015, September). User reviews matter! Tracking crowdsourced reviews to support evolution of successful apps. In 2015 IEEE international conference on software maintenance and evolution (ICSME) (pp. 291-300). Piscataway, NJ: IEEE.
Pappas, I. O., Papavlasopoulou, S., Mikalef, P., & Giannakos, M. N. (2020). Identifying the combinations of motivations and emotions for creating satisfied users in SNSs: An fsQCA approach. International Journal of Information Management, 53. doi:10.1016/j.ijinfomgt.2020.102128
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing, 64(1), 12-37.
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1989). A conceptual model of service quality and its implications for future research. Journal of Marketing, 49(4), 41-50.
Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). ES-QUAL: A multiple-item scale for assessing electronic service quality. Journal of Service Research, 7(3), 213-233.
Park, H., & Lee, T. (2018). Adoption of E-government applications for public health risk communication: Government trust and social media competence as primary drivers. Journal of Health Communication, 23(8), 712-723. doi:10.1080/10810730.2018.1511013
Prakash, A. V., Das, S., & Pillai, K. R. (2021). Understanding digital contact tracing app continuance: Insights from India. Health Policy and Technology, 10(4). doi:10.1016/j.hlpt.2021.100573
Qureshi, A. A., Ahmad, M., Ullah, S., Yasir, M. N., Rustam, F., & Ashraf, I. (2023). Performance evaluation of machine learning models on large dataset of android applications reviews. Multimedia Tools and Applications, 82(24), 37197-37219.
Radovanovic, D., Hoist, C., Belur, S. B., Srivastava, R., Houngbonon, G. V., Le Quentrec, E., . . . Noll, J. (2020). Digital literacy key performance indicators for sustainable development. Social Inclusion, 8(2), 151-167. doi:10.17645/si.v8i2.2587
Ramzy, M., & Ibrahim, B. (2024). User satisfaction with Arabic COVID-19 apps: Sentiment analysis of users' reviews using machine learning techniques. Information Processing & Management, 61(3). doi:10.1016/j.ipm.2024.103644
Sharma, S. K., Al-Badi, A., Rana, N. P., & Al-Azizi, L. (2018). Mobile applications in government services (mG-App) from user's perspectives: A predictive modelling approach. Government Information Quarterly, 35(4), 557-568. doi:10.1016/j.giq.2018.07.002
Sharma, S. K., Metri, B., Dwivedi, Y. K., & Rana, N. P. (2021). Challenges common service centers (CSCs) face in delivering e-government services in rural India. Government Information Quarterly, 38(2). doi:10.1016/j.giq.2021.101573
Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S. F., & Pantic, M. (2017). A survey of multimodal sentiment analysis. Image and Vision Computing, 65, 3-14. doi:10.1016/j.imavis.2017.08.003
Song, Z., Wang, C., & Bergmann, L. (2020). China's prefectural digital divide: Spatial analysis and multivariate determinants of ICT diffusion. International Journal of Information Management, 52. doi:10.1016/j.ijinfomgt.2020.102072
UN E-Government Survey 2022. (2022). Retrieved from https://publicadministration.un.org/egovkb/en-us/Reports/UN-E-government-Survey-2022
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. doi:10.2307/30036540
Wang, C. (2014). Antecedents and consequences of perceived value in Mobile Government continuance use: An empirical research in China. Computers in Human Behavior, 34, 140-147. doi:10.1016/j.chb.2014.01.034
Wang, C., & Teo, T. S. H. (2020). Online service quality and perceived value in mobile government success: An empirical study of mobile police in China. International Journal of Information Management, 52. doi:10.1016/j.ijinfomgt.2020.102076
Wang, C., Teo, T. S. H., Dwivedi, Y., & Janssen, M. (2021). Mobile services use and citizen satisfaction in government: Integrating social benefits and uses and gratifications theory. Information Technology & People, 34(4), 1313-1337. doi:10.1108/itp-02-2020-0097
Wang, M., Zhou, J., & Lin, H. Z. (2021). A sentiment analysis of the influence of service attributes on consumer satisfaction. Journal of Intelligent & Fuzzy Systems, 40(6), 10507-10522. doi:10.3233/jifs-201207
Weerakkody, V., Irani, Z., Lee, H., Hindi, N., & Osman, I. (2016). Are UK citizens satisfied with e-government services? Identifying and testing antecedents of satisfaction. Information Systems Management, 33(4), 331-343. doi:10.1080/10580530.2016.1220216
Weiss, H. M., & Cropanzano, R. (1996). Affective events theory. Research in Organizational Behavior, 18(1), 1-74.
Wen, J., Hussain, H., Jiang, R. A., & Waheed, J. (2023). Overcoming the digital divide with ICT diffusion: Multivariate and spatial analysis at China's provincial level. Sage Open, 13(1). doi:10.1177/21582440231159323
Wong, S. L., Liu, H. T., & Cheng, L. J. (2011). Elucidating the relationship between satisfaction and citizen involvement in public administration. Public Management Review, 13(4), 595-618.
Wu, Y., Ye, Q., Shen, F., Zhang, Z., & Jiang, C. L. (2022). Country- and app-level factors affecting the adoption and evaluation of COVID-19 mobile apps. BMC Public Health, 22(1). doi:10.1186/s12889-022-14918-8
Xiong, L., Wang, H. C., & Wang, C. W. (2022). Predicting mobile government service continuance: A two-stage structural equation modeling-artificial neural network approach. Government Information Quarterly, 39(1). doi:10.1016/j.giq.2021.101654
Yue, L., Chen, W. T., Li, X., Zuo, W. L., & Yin, M. H. (2019). A survey of sentiment analysis in social media. Knowledge and Information Systems, 60(2), 617-663. doi:10.1007/s10115-018-1236-4
Zecevic, M., Mijatovic, D., Koklic, M. K., Zabkar, V., & Gidakovic, P. (2021). User perspectives of diet-tracking apps: Reviews content analysis and topic modeling. Journal of Medical Internet Research, 23(4). doi:10.2196/25160
Zhai, Y. K., Song, X., Chen, Y. J., & Lu, W. (2022). A study of mobile medical app user satisfaction incorporating theme analysis and review sentiment tendencies. International Journal of Environmental Research and Public Health, 19(12). doi:10.3390/ijerph19127466
Zhang, H. Y., & Zhu, Z. J. (2024). Research on citizens' intentions for continued usage of mobile government services from the PPM perspective. Systems, 12(11). doi:10.3390/systems12110488
Zhang, J., Wan, Y., & Xie, T. (2019). Software feature refinement prioritization based on online user review mining. Information and Software Technology, 108, 30-34. doi:10.1016/j.infsof.2018.12.002
Zhu, J., & Hou, H. (2021). Research on user experience evaluation of mobile applications in government services. IEEE Access, 9, 52634-52641. doi:10.1109/access.2021.3070365
Zhu, Y. Q., & Alamsyah, N. (2022). Citizen empowerment and satisfaction with smart city app: Findings from Jakarta. Technological Forecasting and Social Change, 174. doi:10.1016/j.techfore.2021.121304
Zhuo, J., Chen, L., & Li, F. (2009, May). ES-QUAL: Its applicability in evaluating e-government web sites service quality. In 2009 International Symposium on Information Engineering and Electronic Commerce (pp. 515-518). Piscataway, NJ: IEEE.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Lex localis - Journal of Local Self-Government
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
