Open Lakehouse Architectures for Secure Critical Infrastructure Data Sharing

Authors

  • Aditya Laljibhai Patel

Keywords:

lakehouse architecture, critical infrastructure, data sharing, data lake, ACID transactions, SCADA security, industrial control systems, attribute-based encryption, homomorphic encryption, blockchain, FAIR principles, cybersecurity governance

Abstract

The environment of critical infrastructure operators, such as energy companies, water providers and manufacturing institutions has been increasingly based on disjointed data warehouses and data lakes, which continue to make it difficult to share data securely, quickly, and in a standard format across organizational lines. This paper combines insights from sixteen peer-reviewed and technical sources to suggest an open lakehouse architecture that brings together transactional reliability, open storage formats, and layered protection from cryptography and distributed-ledgers to meet the security and interoperability requirements of critical infrastructure operators. It is based on empirical experience with Delta Lake ACID table storage, survey data on supervisory control and data acquisition (SCADA) and industrial control system (ICS) threats, and comparative work on attribute-based encryption, homomorphic encryption, and blockchain based access control. Results show that lakehouse builds based on atomicity, consistency, isolation and durability (ACID) transactions improved the read throughput by about 60 percent compared with standard Hive-based warehousing builds, while permissioned blockchain ledgers maintained a transaction rate of more than 195 transactions per second on 2,000 connected devices. The number of reported ICS and SCADA security incidents has increased from 145 in 2015 to 336 in 2020, highlighting the need for cryptographic and governance controls to be integrated directly in the storage layer. The proposed architecture includes a metadata governance layer based on FAIR (Findable, Accessible, Interoperable, Reusable) principles, a cryptographic layer with attribute-based and homomorphic encryption, and a blockchain trust layer to ensure auditable access control. This includes decreased reporting time for regulators, enhanced interoperability between agencies, measurable decreases in unauthorized access incidents, and recognizing computational cost and adoption challenges detailed in the synthesized literature.

Downloads

Download data is not yet available.

References

Acar, A., Aksu, H., Uluagac, A. S., & Conti, M. (2018). A survey on homomorphic encryption schemes: Theory and implementation. ACM Computing Surveys, 51(4), Article 79. https://doi.org/10.1145/3214303

Armbrust, M., Das, T., Sun, L., Yavuz, B., Zhu, S., Murthy, M., Torres, J., van Hovell, H., Ionescu, A., Łuszczak, A., & Zaharia, M. (2020). Delta Lake: High-performance ACID table storage over cloud object stores. Proceedings of the VLDB Endowment, 13(12), 3411–3424. https://doi.org/10.14778/3415478.3415560

Armbrust, M., Ghodsi, A., Xin, R., & Zaharia, M. (2021). Lakehouse: A new generation of open platforms that unify data warehousing and advanced analytics. In Proceedings of the 11th Conference on Innovative Data Systems Research (CIDR 2021). http://cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf

Barrett, M. P. (2018). Framework for improving critical infrastructure cybersecurity, version 1.1. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.CSWP.04162018

Bhamare, D., Zolanvari, M., Erbad, A., Jain, R., Khan, K., & Meskin, N. (2020). Cybersecurity for industrial control systems: A survey. Computers & Security, 89, Article 101677. https://doi.org/10.1016/j.cose.2019.101677

Ghosh, S., & Sampalli, S. (2019). A survey of security in SCADA networks: Current issues and future challenges. IEEE Access, 7, 135812–135831. https://doi.org/10.1109/ACCESS.2019.2926441

Hai, R., Quix, C., & Jarke, M. (2021). Data lake concept and systems: A survey. arXiv. https://doi.org/10.48550/arXiv.2106.09592

Latif, S., Idrees, Z., Ahmad, J., Zheng, L., & Zou, Z. (2021). A blockchain-based architecture for secure and trustworthy operations in the industrial internet of things. Journal of Industrial Information Integration, 21, Article 100190. https://doi.org/10.1016/j.jii.2020.100190

Liu, H., Han, D., & Li, D. (2020). Fabric-IoT: A blockchain-based access control system in IoT. IEEE Access, 8, 18207–18218. https://doi.org/10.1109/ACCESS.2020.2968492

Lu, Y., Huang, X., Dai, Y., Maharjan, S., & Zhang, Y. (2020). Blockchain and federated learning for privacy-preserved data sharing in industrial IoT. IEEE Transactions on Industrial Informatics, 16(6), 4177–4186. https://doi.org/10.1109/TII.2019.2942190

Pliatsios, D., Sarigiannidis, P., Lagkas, T., & Sarigiannidis, A. G. (2020). A survey on SCADA systems: Secure protocols, incidents, threats and tactics. IEEE Communications Surveys & Tutorials, 22(3), 1942–1976. https://doi.org/10.1109/COMST.2020.2987688

Rahman, Z., Khalil, I., Yi, X., & Atiquzzaman, M. (2021). Blockchain-based security framework for a critical industry 4.0 cyber-physical system. IEEE Communications Magazine, 59(5), 128–134. https://doi.org/10.1109/MCOM.001.2000679

Sawadogo, P., & Darmont, J. (2021). On data lake architectures and metadata management. Journal of Intelligent Information Systems, 56(1), 97–120. https://doi.org/10.1007/s10844-020-00608-7

Tabrizchi, H., & Kuchaki Rafsanjani, M. (2020). A survey on security challenges in cloud computing: Issues, threats, and solutions. The Journal of Supercomputing, 76(12), 9493–9532. https://doi.org/10.1007/s11227-020-03213-1

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., Gonzalez-Beltran, A., Gray, A. J. G., Groth, P., Goble, C., Grethe, J. S., Heringa, J., … Mons, B. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, Article 160018. https://doi.org/10.1038/sdata.2016.18

Zhang, Y., Deng, R. H., Xu, S., Sun, J., Li, Q., & Zheng, D. (2020). Attribute-based encryption for cloud computing access control: A survey. ACM Computing Surveys, 53(4), Article 83. https://doi.org/10.1145/3398036

Downloads

Published

28.08.2022

How to Cite

Aditya Laljibhai Patel. (2022). Open Lakehouse Architectures for Secure Critical Infrastructure Data Sharing. International Journal of Intelligent Systems and Applications in Engineering, 10(3s), 586–596. Retrieved from https://www.ijisae.org/index.php/IJISAE/article/view/8477

Issue

Vol. 10 No. 3s (2022)

Section

Research Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.

IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.