From Episodic Audits to Continuous Intelligence: A Socio-Technical Framework for Enterprise Service Quality Evaluation
Keywords:
Continuous Quality Evaluation, Enterprise Service Operations, Human-in-the-loop Systems, Trustworthy Artificial Intelligence, Socio-technical systems designAbstract
Enterprise service organizations depend on quality evaluation to ensure agent performance and consistent customer experience․ However, it still relies on sample-based, manual processes that assess fewer than five percent of service interactions. Evaluator score variance also ranged from a low of 15 percentage points to a high of 22 percentage points, with feedback times across conditions ranging from a low of 72 hours to a high of 96 hours. This article introduces Continuous Quality Intelligence (CQI), a socio-technical framework for quality as an intelligence-based operating capability. The framework was constructed through a review of service quality management‚ socio-technical systems design‚ and explainable and trustworthy artificial intelligence governance․ Main findings note that full-coverage automated evaluation eliminates sampling bias; configurable multidimensional scoring captures interaction heterogeneity lost by unified models; and structured human calibration improves defect detection by 34 to 47 percent over fully automated evaluation. Governance controls such as explainability, audit logging, and fairness monitoring ensure that scalable evaluation and human accountability are co-evolving properties of a principled socio-technical architecture, creating a regime of continuous quality intelligence at the enterprise scale.
Downloads
References
- Kulkov I, Kulkova J, Rohrbeck R, Menvielle L, Kaartemo V, Makkonen H. Artificial intelligence‐driven sustainable development: Examining organizational, technical, and processing approaches to achieving global goals. Sustainable Development. 2024 Jun;32(3):2253-67.
- Yeğin T, Ikram M. Developing a sustainable omnichannel strategic framework toward circular revolution: an integrated approach. Sustainability. 2022 Sep 15;14(18):11578.
- Kumar S, Datta S, Singh V, Datta D, Singh SK, Sharma R. Applications, challenges, and future directions of human-in-the-loop learning. IEEE Access. 2024 May 15;12:75735-60.
- Ahangar MN, Farhat ZA, Sivanathan A, Ketheesram N, Kaur S. Explainable AI-driven quality and condition monitoring in smart manufacturing. Sensors. 2026 Jan 30;26(3):911.
- Mejia J, Mankad S, Gopal A. Service quality using text mining: Measurement and consequences. Manufacturing & Service Operations Management. 2021 Nov;23(6):1354-72.
- Kim L, Maijan P, Yeo SF. Developing customer service quality: Influences of job stress and management process alignment in banking industry. Sustainable Futures. 2024 Dec 1;8:100311.
- Wirtz J, Lwin MO, Williams JD. Causes and consequences of consumer online privacy concern. International Journal of service industry management. 2007 Aug 14;18(4):326-48.
- Lemon KN, Verhoef PC. Understanding customer experience throughout the customer journey. Journal of marketing. 2016 Nov;80(6):69-96.
- Misra V. Explainable Generative AI for Enterprise CRM Analytics: Interpretable Machine Learning Models for Customer Trust, Compliance, and Ethical AI Governance. International Journal of Technology, Management and Humanities. 2025 Nov 12;11(04):101-14.
- Korzynski P, Paniagua J, Rodriguez-Montemayor E. Employee creativity in a digital era: the mediating role of social media. Management Decision. 2020;58(6):1100-17.
-Baxter G, Sommerville I. Socio-technical systems: From design methods to systems engineering. Interacting with Computers. 2011;23:4-17.
- Mukesh A. AI-Powered Data Engineering Frameworks for Smart Manufacturing Quality Control. International Journal of Engineering & Extended Technologies Research (IJEETR). 2024 Dec 23;6(6):9189-206.
- Basir OA. The Social Responsibility Stack: A Control-Theoretic Architecture for Governing Socio-Technical AI. arXiv preprint arXiv:2512.16873. 2025 Dec 18.
- Sangdean T, Nassehi A, Qi Q. Opportunities of explainable AI for enhancing quality management systems. InMATEC Web of Conferences 2025 (Vol. 413, p. 07002). EDP Sciences.
- Owolabi B. Explainable AI (XAI) in Enterprise Decision Support Systems. Enterprise Decision Support Systems (February 23, 2025). 2025 Feb 23.
- Franciosa P, Sokolov M, Sinha S, Sun T, Ceglarek D. Deep learning enhanced digital twin for Closed-Loop In-Process quality improvement. CIRP annals. 2020 Jan 1;69(1):369-72.
- Kaur D, Uslu S, Rittichier KJ, Durresi A. Trustworthy artificial intelligence: a review. ACM computing surveys (CSUR). 2022 Jan 18;55(2):1-38.
- Islam S, Basheer N, Chakraborty A, Papastergiou S, Lekidis A. Integrated Framework with Fairness and Explainable Ai Practice for Ai-Enabled Software Systems. Available at SSRN 5290734.
- Kumar S, Datta S, Singh V, Datta D, Singh SK, Sharma R. Applications, challenges, and future directions of human-in-the-loop learning. IEEE Access. 2024 May 15;12:75735-60.
- Morley J, Kinsey L, Elhalal A, Garcia F, Ziosi M, Floridi L. Operationalising AI ethics: barriers, enablers and next steps. AI & SOCIETY. 2023 Feb;38(1):411-23.
- Masoudi M. Algorithmic Governance, Data-Driven Decision Making, and the Transformation of Democratic Accountability in Contemporary States. Advanced Journal of Management, Humanity and Social Science. 2025 Jan 21;2(1):10-22.
- Yang W, Li S, Luo G, Li H, Wen X. A Real-Time Human–Machine–Logistics Collaborative Scheduling Method Considering Workers’ Learning and Forgetting Effects. Applied System Innovation. 2025 Mar 18;8(2):40.
- Lazaros K, Vrahatis AG, Kotsiantis S. Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications. Entropy. 2026 Mar 26;28(4):377.
- Kim Y. OntoMotoOS: Toward an Ethical and Evolving Framework for Collective AI Governance.
- Dubey S. From Test Case Design to Test Data Generation: How AI Is Transforming End-to-End Quality Assurance in Agile and DevOps Environments. Authorea Preprints. 2025 Oct 22.
- Hrytsenko O, Kovalchuk I, Petrenko M. AI-Native Decision Support for Cyber-Physical Production: Quality Assurance and Lifecycle Controls. The Artificial Intelligence Journal. 2022 Dec 18;3(4).
- Ribeiro MT, Singh S, Guestrin C. " Why should i trust you?" Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016 Aug 13 (pp. 1135-1144).
- Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Advances in neural information processing systems. 2017;30.
- Amershi S, Weld D, Vorvoreanu M, Fourney A, Nushi B, Collisson P, Suh J, Iqbal S, Bennett PN, Inkpen K, Teevan J. Guidelines for human-AI interaction. InProceedings of the 2019 chi conference on human factors in computing systems 2019 May 2 (pp. 1-13).
- Doshi-Velez F, Kim B. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. 2017 Feb 28.
- Samek W, Montavon G, Lapuschkin S, Anders CJ, Müller KR. Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE. 2021 Mar 4;109(3):247-78.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
