Comparing Agentic Generative UI and Traditional Interfaces: A Controlled Study of Goal-Based AI-Generated Interfaces in Transactional Workflows
Keywords:
Agentic user experience, Generative user interface, Human–AI interaction, Transactional workflows, Trust in automation, UsabilityAbstract
The advent of agentic interfaces is allowing artificial intelligence to create user interfaces on the fly based on layout designs that are appropriate for the user’s purpose and not for a predefined interface design created at the time of design. One practical issue then arises for product teams: does an agentic interface designed based on the user’s purposes lag behind a conventional interface in terms of efficiency in a transactional process? In this research paper, results from a controlled between-subjects experiment are presented. The experiment included a multi-step process of booking the cheapest flight that arrived prior to a particular time with only one stopover. The experiment involved 52 subjects performing this task using either a traditional production-style interface or an agentically generated interface that varied how the same flight information was presented while preserving the complete dataset and all user decisions. The following measures were used to assess the time taken to complete the task, clicks made, back navigation count, perceived workload, usability, trust, perceived control, and satisfaction. The two interfaces did not differ in any of the measures mentioned, and all the subjects completed the task successfully. An analysis of equivalence using two one-sided tests with a margin of ±0.5 standard deviations proved equivalence only in terms of clicks and back navigation count, whereas experiential measures, including trust and perceived workload, were inconclusive, showing neither a reliable difference nor confirmed equivalence. These findings show that there is no disadvantage associated with using the agentically generated interface compared with a proven interface, provided that the information completeness and decision-making control of the user are maintained. The pertinent question is not whether agentically generated interfaces outperform mature interfaces but whether there is any disadvantage associated with their use; here, none was detectable.
Downloads
References
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in Proc. 11th Int. Conf. Learning Representations (ICLR), 2023.
T. Schick, J. Dwivedi-Yu, R. Dessì, et al., “Toolformer: Language models can teach themselves to use tools,” in Advances in Neural Information Processing Systems 36 (NeurIPS), 2023.
S. Yao, H. Chen, J. Yang, and K. Narasimhan, “WebShop: Towards scalable real-world web interaction with grounded language agents,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, 2022.
X. Deng, Y. Gu, B. Zheng, et al., “Mind2Web: Towards a generalist agent for the web,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 36, Datasets and Benchmarks Track, 2023.
S. Zhou, F. F. Xu, H. Zhu, et al., “WebArena: A realistic web environment for building autonomous agents,” in Proc. 12th Int. Conf. Learn. Represent. (ICLR), 2024.
X. Liu, H. Yu, H. Zhang, et al., “AgentBench: Evaluating LLMs as agents,” in Proc. 12th Int. Conf. Learn. Represent. (ICLR), 2024.
T. Xie, D. Zhang, J. Chen, et al., “OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments,” in Advances in Neural Information Processing Systems 38, Datasets and Benchmarks Track (NeurIPS), 2024.
S. Yao, N. Shinn, P. Razavi, and K. Narasimhan, “τ-bench: A benchmark for tool–agent–user interaction in real-world domains,” in Proc. 13th Int. Conf. Learning Representations (ICLR), 2025.
J. Chen, Y. Zhang, Y. Zhang, Y. Shao, and D. Yang, “Generative interfaces for language models,” arXiv:2508.19227, 2025.
Y. Cao, B. Min, A. Chen, and H. Xia, “Generative and malleable user interfaces with generative and evolving task-driven data models,” in Proc. CHI Conf. Human Factors Comput. Syst. (CHI), 2025, doi: 10.1145/3706598.3713285.
B. Min, A. Chen, Y. Cao, and H. Xia, “Malleable overview-detail interfaces,” in Proc. CHI Conf. Human Factors Comput. Syst. (CHI), Art. no. 688, 2025, doi: 10.1145/3706598.3714164.
P. Vaithilingam, E. L. Glassman, J. P. Inala, and C. Wang, “DynaVis: Dynamically synthesized UI widgets for visualization editing,” in Proc. 2024 CHI Conf. Human Factors in Computing Systems (CHI ’24), Art. 985, ACM, 2024, doi: 10.1145/3613904.3642639.
N. Hojo et al., “GenerativeGUI: Dynamic GUI Generation Leveraging LLMs for Enhanced User Interaction on Chat Interfaces,” in Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, in CHI EA ’25. New York, NY, USA: Association for Computing Machinery, 2025. doi: 10.1145/3706599.3719743.
S. Lindley et al., “What does generative UI mean for HCI practice?”, presented at the CHI 2026 Workshop on Generative UI, 2026.
S. Yun, H. Lin, R. Thushara, et al., “Web2Code: A large-scale webpage-to-code dataset and evaluation framework for multimodal LLMs,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 37, Datasets and Benchmarks Track, 2024.
Z. Tang, C. Wu, J. Li, and N. Duan, “LayoutNUWA: Revealing the hidden layout expertise of large language models,” in Proc. 12th Int. Conf. Learn. Represent. (ICLR), 2024.
J. Wu, E. Schoop, A. Leung, T. Barik, J. P. Bigham, and J. Nichols, “UICoder: Fine-tuning large language models to generate user interface code through automated feedback,” in Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics: Hum. Lang. Technol. (NAACL-HLT), 2024, pp. 7511–7525, doi: 10.18653/v1/2024.naacl-long.417.
J. Yoon, J. Cho, J. Kim, J. Chung, J. Jeon, and Y. Yu, “A11YN: Aligning LLMs for Accessible Web UI Code Generation,” 2025.
K. Z. Gajos, D. S. Weld, and J. O. Wobbrock, “Automatically generating personalized user interfaces with SUPPLE,” Artif. Intell., vol. 174, no. 12–13, pp. 910–950, 2010, doi: 10.1016/j.artint.2010.05.005.
K. Z. Gajos, J. O. Wobbrock, and D. S. Weld, “Automatically generating user interfaces adapted to users’ motor and vision capabilities,” in Proc. 20th ACM Symp. User Interface Softw. Technol. (UIST), 2007, pp. 231–240, doi: 10.1145/1294211.1294253.
J. D. Lee and K. A. See, “Trust in automation: Designing for appropriate reliance,” Hum. Factors, vol. 46, no. 1, pp. 50–80, 2004, doi: 10.1518/hfes.46.1.50.30392.
R. Parasuraman and V. Riley, “Humans and automation: Use, misuse, disuse, abuse,” Hum. Factors, vol. 39, no. 2, pp. 230–253, 1997, doi: 10.1518/001872097778543886.
E. Horvitz, “Principles of mixed-initiative user interfaces,” in Proc. SIGCHI Conf. Human Factors Comput. Syst. (CHI), 1999, pp. 159–166, doi: 10.1145/302979.303030.
S. Amershi, D. Weld, M. Vorvoreanu, et al., “Guidelines for human-AI interaction,” in Proc. CHI Conf. Human Factors Comput. Syst. (CHI), 2019, doi: 10.1145/3290605.3300233.
B. Shneiderman, Human-Centered AI. Oxford, U.K.: Oxford Univ. Press, 2022, doi: 10.1093/oso/9780192845290.001.0001.
S. G. Hart and L. E. Staveland, “Development of NASA-TLX: Results of empirical and theoretical research,” in Human Mental Workload, P. A. Hancock and N. Meshkati, Eds. Amsterdam, The Netherlands: Elsevier, 1988, pp. 139–183, doi: 10.1016/S0166-4115(08)62386-9.
J. R. Lewis, B. S. Utesch, and D. E. Maher, “UMUX-LITE: When there’s no time for the SUS,” in Proc. SIGCHI Conf. Human Factors Comput. Syst. (CHI), 2013, pp. 2099–2102, doi: 10.1145/2470654.2481287.
Google LLC, “Google Flights.” [Online]. Available: https://www.google.com/travel/flights. Accessed: Jun. 15, 2026.
MUI, “Material UI – React component library.” [Online]. Available: https://mui.com/material-ui/. Accessed: Jun. 15, 2026.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
