Ahmad-Reza Sadeghi

Mr. Sadeghi, what new challenges do AI systems pose for IT security?

Ahmad-Reza Sadeghi: Algorithms or AI-based systems are fragile from a security perspective because they are very data-dependent. They can be easily manipulated, especially covertly. The more advanced the systems become, the more advanced the attacks are. The biggest risk is our application behavior: If AI systems one day really automate large parts of our everyday lives and make decisions for us, our dependence on these systems will also be much greater. So potential attackers can also do much greater damage. Another challenge is that common IT security systems cannot be easily applied to AI systems. In addition, you don't want to limit the performance of the models with security measures.

How can we protect AI systems and their data from attacks?

Ahmad-Reza Sadeghi: You have to find new ways to secure AI algorithms. In my research, I have also been heavily involved with applied cryptography, i.e. computing with encrypted data. The purely cryptographic solutions are not yet scalable, especially for huge AI models with billions of parameters in some cases. So algorithmic improvements as well as hardware-based solutions for AI security are also currently being researched. Another interesting field is the use of AI for safety-critical systems, i.e. algorithms that protect against attacks.

In terms of data protection, distributed machine learning represents an important opportunity. Here, each end device accesses the respective current model and trains it locally with its own data set. Possible personal data thus does not have to be sent via a central server. Among other things, this increases privacy, for example in a medical context. Hospitals do not share medical data with each other, but can still use distributed machine learning to train the same systems with their data and thus collaborate. However, more points of attack also exist when data and AI models are distributed across more systems. Individual computers could be brought under control, via software or because people in an institution collaborate with the attackers, for example. If this happens, the overall model can be manipulated.

What needs to happen to ensure security in the AI age?

Ahmad-Reza Sadeghi: We need to define the notion of security in the AI context more broadly than before. AI decisions have a high reputation and are often seen as neutral and unbiased. However, they often reflect only the data used to train AI systems - and thus human behavior, habits, and biases. This shows that more attention needs to be paid to social factors in the development of AI systems. The impact of AI systems on our society also needs to be studied in more detail. While AI applications for the financial market, in medicine, or in the legal field are obviously recognizable as critical applications that need to be extensively analyzed and reviewed, consequences of other AI applications such as the recommendation algorithms of Facebook, Twitter, and Google can easily be overlooked. These are the echo chambers that are transforming our societies. I don't worry about terminators, but about the insidious impact of social media on democratic countries and their electoral systems. AI holds many opportunities for business and society. But we will only be able to realize its full potential if we develop and deploy the technology in a secure, privacy-protecting, and ethically responsible way.