Adaptive Frontend Architectures for Real-Time Multi- Source Streaming Dashboards in Large-Scale Web Systems
Keywords:
Adaptive Architecture, Real-Time Streaming, Dashboard Visualization, Frontend Engineering, Web Performance, Large-Scale Systems, Reactive Programming, Data VisualizationAbstract
Modern large-scale web systems—spanning cloud infrastructure monitoring, financial trading platforms, Internet-of-Things (IoT) command centers, and real-time analytics dashboards— demand frontend architectures capable of ingesting, processing, and
rendering data from hundreds of concurrent streaming sources while maintaining interactive frame rates and sub- second visual latency. Existing frontend frameworks treat streaming data consumption as an application-level concern, forcing developers to manually orchestrate connection management, backpressure handling, rendering prioritization, and layout adaptation, leading to brittle architectures that degrade unpredictably under load. This paper introduces StreamAdapt, an adaptive frontend architecture framework purpose-built for real-time multi-source streaming dashboards. StreamAdapt comprises four synergistic components: (1) a Stream Multiplexer that normalizes heterogeneous data sources (WebSocket, Server-Sent Events, gRPC-Web, MQTT-over-WebSocket) into a unified reactive stream abstraction; (2) a Priority-Aware Scheduler that dynamically allocates rendering budget across dashboard widgets based on data criticality, viewport visibility, and user attention signals; (3) an Adaptive Layout Engine that reconfigures dashboard topology in response to stream health, device capabilities, and cognitive load thresholds; and (4) a Progressive Rendering Pipeline that employs level-of-detail rendering, temporal decimation, and speculative pre-rendering to maintain 60 fps under extreme data throughput. We evaluate StreamAdapt against four baseline architectures across three production- representative workloads (infrastructure monitoring with 500 concurrent metric streams, financial market data with 1,200 ticker feeds, and IoT sensor networks with 2,000 device telemetry channels). Results demonstrate that StreamAdapt sustains 58.7 fps median frame rate under 500 concurrent streams (vs. 23.4 fps for React-baseline), reduces 95th- percentile visual latency from 847 ms to 112 ms, and decreases frontend memory consumption by 41.3%. A controlled user study (N = 144) confirms that StreamAdapt dashboards improve anomaly detection accuracy by 28.6% and reduce mean detection time by 34.2% compared to conventional dashboard architectures. This work establishes adaptive frontend architecture as a first-class systems concern for streaming-intensive web applications.
Downloads
References
Angular. (2024). Angular framework documentation. https://angular.dev/.
Akidau, T., Bradshaw, R., Chambers, C., Chernyak, S., Fernández-Moctezuma, R.J., Lax, R., McVeety, S., Mills, D., Perry, F., Schmidt, E., & Whittle, S. (2015). The dataflow model: A practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proceedings of the VLDB Endowment, 8(12), 1792–1803.
Bach, B., Freeman, E., Abdul-Rahman, A., Turkay, C., Khan, S., Fan, Y., & Chen, M. (2023). Dashboard design patterns. IEEE Transactions on Visualization and Computer Graphics, 29(1), 459–469.
Bainomugisha, E., Carreton, A.L., van Cutsem, T., Mostinckx, S., & de Meuter, W. (2013).
A survey on reactive programming. ACM Computing Surveys, 45(4), 1–34.
Battle, L., & Stonebraker, M. (2020). Database-accelerated visualization and analytics.
Proceedings of the VLDB Endowment, 13(12), 3214–3217.
Bostock, M., Ogievetsky, V., & Heer, J. (2011). D3: Data-driven documents. IEEE Transac- tions on Visualization and Computer Graphics, 17(12), 2301–2309.
Brooke, J. (1996). SUS: A quick and dirty usability scale. Usability Evaluation in Industry, 189, 4–7.
Budish, E., Cramton, P., & Shim, J. (2015). The high-frequency trading arms race: Frequent batch auctions as a market design response. The Quarterly Journal of Economics, 130(4), 1547–1621.
Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., & Tzoumas, K. (2015). Apache Flink: Stream and batch processing in a single engine. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 38(4), 28–38.
Chen, J., Chen, B., & Qu, H. (2019). DataSite: Proactive visual data exploration with computation of insight-based recommendations. Information Visualization, 18(2), 251–267.
Cheng, B., Longo, S., Cirillo, F., Bauer, M., & Kovacs, E. (2022). Building a big data platform for smart cities: Experience and lessons from Santander. IEEE Internet of Things Journal, 9(10), 7378–7390.
Fette, I., & Melnikov, A. (2011). The WebSocket protocol. RFC 6455, IETF.
Findlater, L., & McGrenere, J. (2004). A comparison of static, adaptive, and adaptable menus.
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 89–96.
Gajos, K., & Weld, D.S. (2004). SUPPLE: Automatically generating user interfaces. Proceed- ings of the 9th International Conference on Intelligent User Interfaces, 93–100.
Gartner, Inc. (2024). Market guide for real-time streaming analytics platforms. Gartner Research Report G00789234.
Grafana Labs. (2024). Grafana documentation. https://grafana.com/docs/grafana/ latest/.
gRPC Authors. (2023). gRPC-Web: A JavaScript implementation of gRPC for browser clients.
https://grpc.io/docs/platforms/web/.
Gutwin, C., Lippold, M., & Graham, T.C.N. (2011). Real-time groupware in the browser: Testing the performance of web-based networking. Proceedings of the ACM Conference on Computer Supported Cooperative Work, 167–176.
Hart, S.G., & Staveland, L.E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139–183.
Hearst, M.A., & Tory, M. (2019). Would you like a chart with that? Incorporating visualiza- tions into conversational interfaces. IEEE Visualization Conference, 1–5.
Hickson, I. (2015). Server-Sent Events. W3C Recommendation.
Kreps, J., Narkhede, N., & Rao, J. (2011). Kafka: A distributed messaging system for log processing. Proceedings of the NetDB Workshop, 1–7.
Kumar, A., Bhatia, M., & Huang, R. (2023). Cognitive load-aware dashboard design: Metrics and optimization strategies. International Journal of Human-Computer Studies, 171, 102976.
Lavie, T., & Meyer, J. (2010). Benefits and costs of adaptive user interfaces. International Journal of Human-Computer Studies, 68(8), 508–524.
Liu, Z., Jiang, B., & Heer, J. (2019). imMens: Real-time visual querying of big data. Computer Graphics Forum, 32(3), 421–430.
Liu, D., Zhao, Y., Xu, H., Sun, Y., Pei, D., Luo, J., Jing, X., & Feng, M. (2023). Practical root cause analysis for microservice-based systems. ACM Transactions on Software Engineering and Methodology, 32(1), 1–37.
Meyerovich, L.A., & Bodik, R. (2010). Fast and parallel webpage layout. Proceedings of the 19th International Conference on World Wide Web, 711–720.
Meyerovich, L.A., & Bodik, R. (2010). Parallel schedule generation for multi-core browsers.
ACM SIGPLAN Notices, 45(6), 303–314.
OASIS Standard. (2019). MQTT version 5.0. https://docs.oasis-open.org/mqtt/mqtt/ v5.0/mqtt-v5.0.html.
Nicoara, A., Chen, M., & Elmqvist, N. (2023). Dashboard performance engineering: Challenges and solutions for real-time monitoring interfaces. IEEE Computer Graphics and Applications, 43(3), 48–58.
Paas, F., Tuovinen, J.E., Tabbers, H., & Van Gerven, P.W.M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63–71.
Panchenko, A., Karatay, B., & Reznik, Y. (2022). Web performance challenges in real-time data streaming applications. Proceedings of the Web Conference 2022, 3124–3133.
Rajkomar, A., Oren, E., Chen, K., Dai, A.M., Hajaj, N., Hardt, M., Liu, P.J., Liu, X., Marcus,
J., Sun, M., Sundberg, P., Yee, H., Zhang, K., Zhang, Y., Flores, G., Duggan, G.E., Irvine,
J., Le, Q., Litsch, K., Mossin, A., Tansuwan, J., Wang, D., Wexler, J., Wilson, J., Ludwig,
D., Volchenboum, S.L., Chou, K., Pearson, M., Madabushi, S., Shah, N.H., Butte, A.J., Howell, M.D., Cui, C., Corrado, G.S., & Dean, J. (2018). Scalable and accurate deep learning with electronic health records. npj Digital Medicine, 1(1), 18.
React. (2024). React documentation. https://react.dev/.
RxJS. (2024). Reactive extensions library for JavaScript. https://rxjs.dev/.
Sarikaya, A., Correll, M., Bartram, L., Tory, M., & Fisher, D. (2018). What do we talk about when we talk about dashboards? IEEE Transactions on Visualization and Computer Graphics, 25(1), 682–692.
Satyanarayan, A., Moritz, D., Wongsuphasawat, K., & Heer, J. (2016). Vega-Lite: A grammar of interactive graphics. IEEE Transactions on Visualization and Computer Graphics, 23(1), 341–350.
Sisinni, E., Saifullah, A., Han, S., Jennehag, U., & Gidlund, M. (2018). Industrial Internet of Things: Challenges, opportunities, and directions. IEEE Transactions on Industrial Informatics, 14(11), 4724–4734.
Steinarsson, S. (2013). Downsampling time series for visual representation. MSc Thesis, University of Iceland.
Svelte. (2024). Svelte documentation. https://svelte.dev/.
Tao, Y., Shi, L., & Chen, W. (2023). Challenges and opportunities in financial data visualiza- tion. Visual Informatics, 7(2), 1–12.
Vue.js. (2024). Vue.js documentation. https://vuejs.org/.
Welch, C., Kim, S., & Johnson, M. (2023). Performance optimization patterns for React applications at scale. Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 1234–1245.
Wu, A., Wang, Y., Shu, X., Moritz, D., Cui, W., Zhang, H., Zhang, D., & Qu, H. (2022). AI4VIS: Survey on artificial intelligence approaches for data visualization. IEEE Transac- tions on Visualization and Computer Graphics, 28(12), 5049–5070.
Zaharia, M., Xin, R.S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M.J., Ghodsi, A., Gonzalez, J., Shenker, S., & Stoica, I. (2016). Apache Spark: A unified engine for big data processing. Communications of the ACM, 59(11), 56–65.
Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Li, W., & Ding, D. (2023). Fault analysis and debugging of microservice systems: Industrial survey, benchmark system, and empirical study. IEEE Transactions on Software Engineering, 49(4), 1834–1862.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
