Introduction

In the realm of future industrial applications, the demand for seamless, efficient wireless connectivity is paramount. To meet this need, innovative approaches to network resource management and spectrum utilization are essential. This article introduces some of the innovations of 6G-BRAINS, which leverages cutting-edge Artificial Intelligence (AI) technologies, such as Multi-Agent Deep Reinforcement Learning (MA-DRL).

Use cases and KPIs

Through meticulous research and stakeholder engagement, the first step identified the sectors, usage scenarios and use cases where these innovations can make the greatest impact. The motivation was to derive a wealth of user requirements and technology Key Performance Indicators (KPIs), which 5G cannot deliver. Among the obvious ambitious KPIs in an industrial environment, like ultra-low latency, the project identified gaps and missing indicators such as location accuracy and direction accuracy, which prompted the exploration of advanced features such as sensing.

Multi-agent Deep Reinforcement Learning

6G-BRAINS made significant advance in the understanding and implementation of Deep Reinforcement Learning (DRL) in the context of 6G networks. At the outset, the project introduced the foundational concepts of Reinforcement Learning (RL), elucidating its formalism and the technical hurdles that must be overcome to integrate it seamlessly into the 6G landscape. Building on this groundwork, the project delved into real-world application cases of RL, viewing them through the network domain modelling as well as the problem formulation.

A pivotal aspect involved the specification of key components comprising a Multi-Agent Deep Reinforcement Learning (MA-DRL) scheme, their generic interfaces and how these components interact through training and inference workflows (Figure 1).

This framework laid the groundwork for a practical implementation and to foster a deeper understanding for describing application cases of RL in 6G and enable implementing RL in the 6G network. The projection of the RL process on 6G allowed for the formalisation of 6G resource allocation and scheduling as RL problems.

Figure 1: Functional diagram of a Reinforcement Learning

Radio Link Control using RL

In a study, based on a 5G New Radio (NR) inter-cell interference downlink model, the project simulated a cell free network and used RL to choose the best modulation and coding scheme from the channel quality indicator. The results can be used to enhance the 5G network model radio link control, using RL and Industry 4.0 traffic models.

D2D-enabled cooperative network for cell-free access

In the RL application case of improving the communication performance of far cell-edge terminals, the project demonstrated Device-to-Device (D2D) relays between terminals and an Integrated Access and Backhaul (IAB) node. The application case demonstrated the dynamic allocation of transmission power levels for far cell-edge users and the D2D relay, as well as distinguishing individual terminals by beamforming and successive interference cancellation. The impact of this, is an improved communication quality of far cell-edge terminals with poor channel conditions, thereby expanding cell coverage.

Include a title Further Innovations include (i) design and prototyping of enabling technologies for end-to-end network slicing across the radio access and core network segments, utilising AI-based radio link control and radio access slice scheduler; (ii) enablers for 3D localisation through sensor data fusion and the application of RL to improve location accuracy; (iii) integration of blockchain location ledger technologies for sharing position data.

Finally, the project performed 3D laser measurements at a factory, obtaining a raytracing model that allows simulations with spatial consistency over different bands, providing an accurate geometrical representation of the environment from the propagation properties for precise localisation applications.

Figure 2: D2D enabled cooperative network model.

Outlook

Having achieved significant milestones in leveraging Artificial Intelligence (AI), particularly Multi-Agent Deep Reinforcement Learning (MA-DRL), in the context of 6G networks, the 6G-BRAINS project sets the stage for future advancements and research directions. These include among others: (i) enhanced AI integration and deployment for demonstrating optimisation of 6G network architecture at large scale, (ii) expansion of use cases to a broader range of applications and vertical sectors; (iii) establishing large data set repositories for training and benchmarking AI/ML algorithms; (iv) identifying potential needs for standardisation and policy, governing the deployment and operation of AI-driven network management and control systems, and (v) developing models to help us resolve the tussle between utility of AI/ML and the energy demand for the application of it. In all these areas, initiatives have started, but must be intensified to meet the challenges of 6G and future generation mobile networks.

Further information

• Project 6G-BRAINS website: https://6g-brains.eu/