Generation of lightweight models for cyber-attacks detection algorithms using knowledge distillation

Abstract

Industry 4.0 paradigm has brought about the changes in the way we manufacture. The integration of Cyber-Physical Systems into the Industrial Internet of Things represents the basis for the transition from traditionally centralized to distributed control systems where the overall control task is achieved through the cooperation of different devices which implies their mutual communication and constant information exchange. However, ubiquitous communication between devices with communication and computation capabilities opens up space for various cyber-attacks which can lead to catastrophic damage to equipment and also can endanger the environment and human lives. Therefore, the development and implementation of cyber-attacks detection mechanisms are necessary to prevent negative effects. Deep learning (DL) techniques are successfully applied to generate models on which cyber-attacks detection algorithms are based. However, the size of the DL models is often unsuitable for implementation on industrial control devices that usually have significant computational constraints. The use of complex DL models may disrupt the operation of control systems and introduce unacceptable delays in real-time cyber-attacks detection algorithms. This paper explores the possibilities for application of knowledge distillation technique to generate lightweight DL models. These models are designed to align with the limitations of the devices on which they are deployed. The paper evaluates the performance of lightweight models in cyber-attacks detection algorithms, and compares them to algorithms based on DL models before distillation.