Channel assignment and power allocation for throughput improvement with PPO in B5G heterogeneous edge networks

Abstract

In Beyond the Fifth Generation (B5G) heterogeneous edge networks, numerous users are multiplexed on a channel or served on the same frequency resource block, in which case the transmitter applies coding and the receiver uses interference cancellation. Unfortunately, uncoordinated radio resource allocation can reduce system throughput and lead to user inequity, for this reason, in this paper, channel allocation and power allocation problems are formulated to maximize the system sum rate and minimum user achievable rate. Since the construction model is non-convex and the response variables are high-dimensional, a distributed Deep Reinforcement Learning (DRL) framework called distributed Proximal Policy Optimization (PPO) is proposed to allocate or assign resources. Specifically, several simulated agents are trained in a heterogeneous environment to find robust behaviors that perform well in channel assignment and power allocation. Moreover, agents in the collection stage slow down, which hinders the learning of other agents. Therefore, a preemption strategy is further proposed in this paper to optimize the distributed PPO, form DP-PPO and successfully mitigate the straggler problem. The experimental results show that our mechanism named DP-PPO improves the performance over other DRL methods.