Information Technology for Assessing and Ensuring Cybersecurity of Large Language Models
DOI:
https://doi.org/10.31861/sisiot2025.2.02020Keywords:
information technology, cybersecurity, Large Language Models, IMECA, countermeasuresAbstract
The rapid evolution of large language models (LLMs) and their incredible ability to work with natural language is generating interest within an increasing number of human activities. Modern language models are no longer limited to simple text generation. They can perform the following complex operational processes: reasoning and planning, content generation and big data processing, programming, and information retrieval. LLMs bring significant benefits to various industries, including finance, education, and the public sector. However, in addition to the significant advantages of using these models, there are certain security challenges that must be taken into account when developing and using LLMs. These challenges include generating incorrect answers (hallucinations), creating forbidden content, and generating responses that contain confidential data. This study presents a software tool and technology for assessing and ensuring the cybersecurity of LLMs against the generation of forbidden content. The main goal of this tool is to improve the accuracy of security assessment and the level of protection of LLMs against this threat. A set of basic data required for the software tool was identified, which includes exploits, prompts for checking the model’s output, and countermeasures for its protection. A procedure for collecting, converting, storing, and potentially extending and adapting this data to the individual requirements of the tool’s users is proposed. A functional model of the technology was developed, which consists of the following stages: environment setup (verification of configuration options, verification of connection with models); analysis of system vulnerabilities by simulating attacks on it and verification of the results of its work; analysis of threats, effects, and criticality of attacks on the system using the IMECA (Intrusion Modes Effects Criticality Analysis) method of assessing LLMs; choice of countermeasures (CM) to ensure the cybersecurity of the system. A test of the software tool was conducted, confirming its effectiveness in increasing the security of LLMs due to more complete and trustworthy assessing effects of attacks on vulnerabilities and choice of justified CM set. Directions for future research on increasing the flexibility and usability of the software tool and technology as a whole were proposed, specifically, managing its settings and extending and adapting the basic dataset to the individual requirements of users.
Downloads
References
R. Azoulay, T. Hirst, and S. Reches, “Large Language Models in Computer Science Classrooms: Ethical Challenges and Strategic Solutions,” Applied Sciences, vol. 15, no. 4, p. 1793, 2025, doi:10.3390/app15041793.
P. S. Papageorgiou, R. C. Christodoulou, R. Pitsillos, V. Petrou, G. Vamvouras, E. V. Kormentza, P. J. Papagelopoulos, and M. F. Georgiou, “The Role of Large Language Models in Improving Diagnostic-Related Groups Assignment and Clinical Decision Support in Healthcare Systems: An Example from Radiology and Nuclear Medicine,” Applied Sciences, vol. 15, no. 16, p. 9005, 2025, doi:10.3390/app15169005.
D. K. C. Lee, C. Guan, Y. Yu, and Q. Ding, “A comprehensive review of generative AI in finance,” FinTech, vol. 3, no. 3, pp. 460–478, 2024, doi:10.3390/fintech3030025.
K. Choutri, S. Fadloun, A. Khettabi, M. Lagha, S. Meshoul, and R. Fareh, “Leveraging Large Language Models for Real-Time UAV Control,” Electronics, vol. 14, no. 21, p. 4312, 2025, doi:10.3390/electronics14214312.
O. Neretin and V. Kharchenko, “A model of ensuring LLM cybersecurity,” Radioelectronic and Computer Systems, vol. 2025, no. 2, pp. 201–215, 2025, doi:10.32620/reks.2025.2.13.
M. M. Billah, H. S. Hamjaya, H. Shiralizade, V. Singh, and R. Inam, “Large Language Models’ Trustworthiness in the Light of the EU AI Act—A Systematic Mapping Study,” Applied Sciences, vol. 15, no. 14. p. 7640, 2025, doi:10.3390/app15147640.
P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V. Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer, and H. Hassani, “Jailbreakbench: An open robustness benchmark for jailbreaking large language models,” arXiv preprint arXiv:2404.01318, 2024, doi:10.48550/arXiv.2404.01318.
X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “”Do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pp. 1671–1685, 2024, doi:10.1145/3658644.3670388.
J. Chu, Y. Liu, Z. Yang, X. Shen, M. Backes, and Y. Zhang, “JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs,” arXiv preprint arXiv:2402.05668, 2024, doi:10.48550/arXiv.2402.05668.
M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basar, B. Li, and D. Forsyth, “Harmbench: A standardized evaluation framework for automated red teaming and robust refusal,” arXiv preprint arXiv:2402.04249, 2024, doi:10.48550/arXiv.2402.04249.
I. Babeshko, O. Illiashenko, V. Kharchenko, and K. Leontiev, “Towards Trustworthy Safety Assessment by Providing Expert and Tool-Based XMECA Techniques,” Mathematics, vol. 10, no. 13, p. 2297, 2022, doi:10.3390/math10132297.
A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does LLM safety training fail?,” arXiv preprint arXiv:2307.02483, 2023, doi:10.48550/arXiv.2307.02483.
Y. Wang, H. Li, X. Han, P. Nakov, and T. Baldwin, “Do-not-answer: A dataset for evaluating safeguards in LLMs,” arXiv preprint arXiv:2308.13387, 2023, doi:10.48550/arXiv.2308.13387.
P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong. “Jailbreaking black box large language models in twenty queries,” in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 23–42, 2025, doi:10.1109/SaTML64287.2025.00010.
A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043, 2023, doi:10.48550/arXiv.2307.15043.
A. Souly, Q. Lu, D. Bowen, T. Trinh, E. Hsieh, S. Pandey, P. Abbeel, J. Svegliat, S. Emmons, O. Watkins, and S. Toyer, “A strongreject for empty jailbreaks,” arXiv preprint arXiv:2402.10260, 2024, doi:10.48550/arXiv.2402.10260.
G. Goren, S. Katz, and L. Wolf, “AlignTree: Efficient Defense Against LLM Jailbreak Attacks,” arXiv preprint arXiv:2511.12217, 2025, doi:10.48550/arXiv.2511.12217.
Y. Zhang, L. Ding, L. Zhang, and D. Tao, “Intention analysis makes llms a good jailbreak defender,” arXiv preprint arXiv:2401.06561, 2024, doi:10.48550/arXiv.2401.06561.
O. Neretin and V. Kharchenko, “Model for describing processes of AI systems vulnerabilities collection and analysis using big data tools,” 2022 12th International Conference on Dependable Systems, Services and Technologies (DESSERT), pp. 1-5, 2022. doi: 10.1109/DESSERT58054.2022.10018811.
Published
Issue
Section
License
Copyright (c) 2025 Security of Infocommunication Systems and Internet of Things
This work is licensed under a Creative Commons Attribution 4.0 International License.
