Optimizing AI/ML Workloads in Cloud Environments: A Scalable Approach

Authors

  • Srinivasa Subramanyam Katreddy

Keywords:

AI/ML Workloads, Cloud Optimization, Scalable Infrastructure, Resource Allocation, Multi-Cloud Strategies.

Abstract

AI/ML workloads present unique challenges in resource-intensive cloud environments, necessitating innovative optimization techniques. This paper introduces a scalable framework for optimizing AI/ML workloads in multi-cloud and hybrid cloud infrastructures. The approach leverages dynamic resource allocation, auto-scaling mechanisms, and workload scheduling algorithms to enhance performance and cost efficiency. Experimental results demonstrate reduced latency, improved throughput, and significant cost savings across diverse AI/ML applications. This work provides actionable insights for organizations aiming to optimize cloud usage for complex AI workloads.

Downloads

Download data is not yet available.

References

Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., ... & Zaharia, M. (2010). A view of cloud computing. *Communications of the ACM*, 53(4), 50-58. https://dl.acm.org/doi/10.1145/1721654.1721672

Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and Abstraction in Sociotechnical Systems. *ACM Conference on Fairness, Accountability, and Transparency*, 59-68. https://dl.acm.org/doi/10.1145/3287560.3287598

Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for YouTube recommendations. *Proceedings of the 10th ACM conference on Recommender Systems*, 191-198. https://dl.acm.org/doi/10.1145/2959100.2959190

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.48550/arXiv.1810.04805

Halevy, A., Norvig, P., & Pereira, F. (2009). The unreasonable effectiveness of data. IEEE Intelligent Systems, 24(2), 8-12. https://doi.org/10.1109/MIS.2009.36

Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and Policy Considerations for Deep Learning in NLP. 57th Annual Meeting of the Association for Computational Linguistics (pp. 3645-3650). https://doi.org/10.48550/arXiv.1906.02243

Voigt, P., & Von dem Bussche, A. (2017). The EU General Data Protection Regulation (GDPR). https://dl.acm.org/doi/10.5555/3152676

Doctor, A. (2023). Manufacturing of Medical Devices Using Artificial Intelligence-Based Troubleshooters. In: Paunwala, C., et al. Biomedical Signal and Image Processing with Artificial Intelligence. EAI/Springer Innovations in Communication and Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-15816-2_11

Preyaa Atri, "Design and Implementation of High-Throughput Data Streams using Apache Kafka for Real-Time Data Pipelines", International Journal of Science and Research (IJSR), Volume 7 Issue 11, November 2018, pp. 1988-1991, https://www.ijsr.net/getabstract.php?paperid=SR24422184316 and Research

Preyaa Atri, "Enhancing Big Data Interoperability: Automating Schema Expansion from Parquet to BigQuery", International Journal of Science (IJSR), Volume https://www.ijsr.net/getabstract.php?paperid=SR24522144712 8 Issue 4, April 2019, pp. 2000-2002,

Atri P. Enabling AI Work flows: A Python Library for Seamless Data Transfer between Elasticsearch and Google Cloud Storage. J Artif Intell Mach Learn & Data Sci 2022, 1(1), 489-491. DOI: doi.org/10.51219/JAIMLD/preyaa-atri/132

M.A., Ferrag "The performance evaluation of blockchain-based security and privacy systems for the internet of things: a tutorial." IEEE Internet of Things Journal 8.24 (2021): 17236-17260.

S., Bauk "Blockchain implementation barriers in maritime: a case study based on ism and micmac techniques." Journal of Maritime Research 20.3 (2023): 72-80.

V.S., Rao "Energy exchange process for smart grid based on integrating blockchain with gcn-lstm." Journal of Theoretical and Applied Information Technology 101.24 (2023): 8430-8446.

X., Lin "Making knowledge tradable in edge-ai enabled iot: a consortium blockchain-based efficient and incentive approach." IEEE Transactions on Industrial Informatics 15.12 (2019): 6367-6378.

D., Folkinshteyn "Braving bitcoin: a technology acceptance model (tam) analysis." Journal of Information Technology Case and Application Research 18.4 (2016): 220-249.

H., Wu "Blockchain-based onsite activity management for smart construction process quality traceability." IEEE Internet of Things Journal 10.24 (2023): 21554- 21565.

R., Pise "A survey on smart contract vulnerabilities and safeguards in blockchain." International Journal of Intelligent Systems and Applications in Engineering 10.3s (2022): 1-16.

J.G., Allen "Wrapped and stacked: 'smart contracts' and the interaction of natural and formal language." European Review of Contract Law 14.4 (2018): 307- 343.

K., Werbach "Contracts ex machina." Duke Law Journal 67.2 (2017): 313-382.

W., Lee "A robust identity recovery scheme for the ethereum blockchain platform." Information (Japan) 20.11 (2017): 8133-8141.

X., Feng "Cobc: a blockchain-based collaborative inference system for internet of things." IEEE Internet of Things Journal 10.24 (2023): 21389-21400.

W.A., Kaal "Crypto transaction dispute resolution." Business Lawyer 73.1 (2017): 109-151.

Y. Jani, A. Jani, and K. Prajapati, “Leveraging multimodal ai in edge computing for real time decisionmaking,”computing, vol. 7, no. 8, pp. 41– 51, 2023.

Rahul Kalva. Revolutionizing healthcare cybersecurity a generative AI-Driven MLOps framework for proactive threat detection and mitigation, World Journal of Advanced Research and Reviews, v. 13, n. 3, p. 577-582, 2022.

Downloads

Published

30.11.2023

How to Cite

Srinivasa Subramanyam Katreddy. (2023). Optimizing AI/ML Workloads in Cloud Environments: A Scalable Approach. International Journal of Intelligent Systems and Applications in Engineering, 11(11s), 710 –. Retrieved from https://www.ijisae.org/index.php/IJISAE/article/view/7378

Issue

Vol. 11 No. 11s (2023)

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.

IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.