AI on the end device: How edge AI ensures data protection and energy efficiency

Using Artificial Intelligence (AI) locally on smartphones, in vehicles or industrial robots offers enormous opportunities for the German economy and society. Compared to large AI models that are operated centrally on large computing infrastructures, edge AI requires significantly less energy, protects the privacy and data of users and enables reliable applications in real time. Plattform Lernende Systeme recognises that Germany is in a good starting position in international competition to exploit the potential of the technology. However, research and development face technical challenges and hurdles when it comes to transferring the technology into practice. A current white paper provides an overview of the strengths and weaknesses of Edge AI and discusses options for putting the technology into practice.

Edge AI can be used to monitor the health of patients in real time. Driver assistance systems can react to obstacles at lightning speed and thus increase road safety. The aim of the technology is to process and analyse data as close as possible to where it is generated, i.e. close to the end device. As the data does not have to be transmitted over long distances to data centres in this way, the AI systems can react more quickly. The data of individuals or companies remains securely with the user. This opens up a wide range of potential applications wherever real-time operation is desirable and sensitive data is processed, such as health data in medicine or valuable company data. However, edge AI systems on many end devices may offer more targets for cyberattacks than a well-protected data centre.

Spectacular generative AI models are currently dominating the public debate. They are based on ever larger volumes of centrally processed data and ever higher computing capacities. Their costs and energy consumption are high: an image generator requires as much energy to create an image as is needed to charge a mobile phone battery. Edge AI has to make do with very limited computing power and storage capacity on the end device and must therefore be particularly energy-efficient. The authors of the white paper consider these limitations to be both a challenge for development and an opportunity, as they are drivers for resource-saving AI innovations.

Edge AI is a technological building block for tackling challenges such as climate change, digital sovereignty and energy supply, according to the white paper. For example, Edge AI-based electricity meters can contribute to a stable supply of renewable energy. Sensors in recycling plants can use the technology to recognise valuable materials in waste. Edge AI enables companies to operate more independently of cloud providers, most of whom are based outside Europe, as data streams are processed at source.

"The advantages mentioned open up a great deal of potential, particularly in leading German industries such as the automotive, mechanical engineering and medical technology sectors. Even if there are already successes to show, we are still far from utilising the available potential. The design of the networks, the training of the networks, the translation of the networks and the hardware architectures for calculating the networks are largely considered independently. A holistic approach is necessary to provide powerful edge AI technology," says Wolfgang Ecker, Distinguished Engineer at Infineon Technologies and member of the Technological Enablers and Data Science working group of Plattform Lernende Systeme. "And technology and applications must also be considered together. Edge AI machines can only be designed efficiently with the knowledge of the application and, in turn, new applications can only be developed with the knowledge of the performance of Edge AI technology."

Strengthening transfer into practice

According to the authors of the white paper, the combination of technological knowledge, expertise in the local industries and experience in the development of physical products makes Germany and Europe an ideal location for exploiting the potential of edge AI. However, the software solutions for a specific piece of hardware cannot usually simply be transferred to another end device. There is also a lack of experts in hardware design. Both of these factors slow down the use of edge AI in practice. The authors recommend developing platforms that provide basic building blocks for Edge AI that can be adapted to different industries as required. This requires appropriate standardisation. Research into resource-saving data processing within the limits of the end devices should also be driven forward.

About the white paper

The white paper "Edge AI: AI close to the end device. Technology for more data protection, energy efficiency and applications in real time" was written by members of Plattform Lernende Systeme. The two working groups Technological Enablers and Data Science as well as IT Security, Privacy, Legal and Ethical Framework were in charge. The white paper is mainly based on the results of a round table with experts held in 2023. The publication is available to download (in German) free of charge.

An interview with Wolfgang Ecker, author of the white paper and member of Plattform Lernende Systeme, is available for editorial use.

More in-depth information on distributed machine learning on edge AI systems can be found in the short publication AI at a glance.