Virtual Network Embedding using a Federated DRL Approach

This paper deals with allocating resources for virtual network requests in the underlying substrate of an infrastructure provider (INP). This is called the virtual network embedding problem. A client's virtual network request (VNR) is accepted if the INP's Virtual Network Embedding (VNE) agent successfully maps all the virtual nodes and links to suitable substrate nodes and links satisfying the CPU and bandwidth requirements of nodes and links. We consider a scenario where multiple INPs exist, catering to the VNR requests of their respective clients. In this paper, we have used a Federated Deep Reinforcement Learning (Federated DRL) approach, where the INPs individually train their agents based on DRL. During the training, the DRL agent learns to map the virtual nodes to substrate nodes. Subsequently, the INPs operators learn collaboratively by sharing only the respective trained model information, without revealing any detailed vital information to each other. The federated reinforcement learning model is implemented using TensorFlow; the VNRs arrival is modeled using SimPy package, and the network graphs for virtual and substrate networks are created using the NetworkX package. The proposed solution accepts around 5% more VNRs, accumulates around 7% more revenue, and performs around 10% better in terms of the revenue to embedding cost ratio compared to existing algorithms.