Systems and programs that rely on neural networks are susceptible to cyber-attacks, including the injection of malicious training data during the training process. This can enable attackers to extract valuable training data and model parameters and can lead to targeted modification of the network behavior, all while being stealthy and difficult to detect. Our researchers have developed a system for securely deploying convolutional neural networks (CNNs) to critical edge applications – such as self-driving cars. The system consists of methods for the detection of and defense against Trojan attacks in the CNN. The technology leverages a process called stochastic parameter mutation to mutate the original CNN into an ecosystem of unique CNNs. The ecosystem is then deployed to the hardware devices, where each device has its own unique variant of the CNN model. This, coupled with a special update mechanism, enables a one-to-one relationship between a model (device) and any updates made to it. If a device receives an update meant for another device, it will not work properly, and the attack will be easily detected. This makes it difficult for an attacker to insert a stealthy Trojan and reduces the likelihood of a Trojan update from spreading to other devices in the ecosystem.
Threat modeling for the Trojan attack