Self-play learning strategies for resource assignment in Open-RAN networks

Xiaoyang Wang; Jonathan D. Thomas; Robert J. Piechocki; Shipra Kapoor; Raúl Santos-Rodríguez; Arjun Parekh

doi:10.1016/j.comnet.2021.108682

Self-play learning strategies for resource assignment in Open-RAN networks

Xiaoyang Wang
, Jonathan D. Thomas
, Robert J. Piechocki^*
, Shipra Kapoor
, Raúl Santos-Rodríguez
, Arjun Parekh

^*Corresponding author for this work

Research output: Contribution to journal › Article (Academic Journal) › peer-review

29 Citations (Scopus)

155 Downloads (Pure)

Abstract

Open Radio Access Network (ORAN) is being developed with an aim to democratise access and lower the cost of future mobile data networks, supporting network services with various QoS requirements, such as massive IoT and URLLC. In ORAN, network functionality is dis-aggregated into remote units (RUs), distributed units (DUs) and central units (CUs), which allows flexible software on Commercial-Off-The-Shelf (COTS) deployments. Furthermore, the mapping of variable RU requirements to local mobile edge computing centres for future centralised processing would significantly reduce the power consumption in cellular networks. In this paper, we study the RU–DU resource assignment problem in an ORAN system, modelled as a 2D bin packing problem. A deep reinforcement learning-based self-play approach is proposed to achieve efficient RU–DU resource management, with AlphaGo Zero inspired neural Monte-Carlo Tree Search (MCTS). Experiments on representative 2D bin packing environment and real sites data show that the self-play learning strategy achieves intelligent RU–DU resource assignment for different network conditions. Comparing with baseline methods, including a heuristic virtual resource allocation algorithm, the Lego heuristic algorithm and the MCTS methods, the proposed approach achieves a performance gain between 5.70% to 12.95% in terms of resource utilisation efficiency.

Original language	English
Article number	108682
Journal	Computer Networks
Volume	206
Early online date	1 Jan 2022
DOIs	https://doi.org/10.1016/j.comnet.2021.108682
Publication status	Published - 7 Apr 2022

Bibliographical note

Funding Information:
This work is funded by the Next-Generation Converged Digital Infrastructure (NG-CDI) Project, supported by BT and Engineering and Physical Sciences Research Council (EPSRC), United Kingdom , Grant ref. EP/R004935/1 .

Funding Information:
Robert J. Piechocki is a Professor of Wireless Systems at the University of Bristol. His research expertise is in the areas of Connected Intelligent Systems, Wireless & Self-Learning Networks, Information and Communication Theory, Statistics and AI. Rob has published over 200 papers in peer-reviewed international journals and conferences and holds 13 patents in these areas. He leads wireless connectivity and sensing research activities for the IRC SPHERE project (winner of 2016 World Technology Award). He collaborates on research grants totalling over £25M, and is a PI/CI for several high-profile projects in networks and AI funded by the industry, EU, Innovate UK and EPSRC such as NG-CDI, OPERA, AIMM, FLOURISH. He regularly advises the industry and the Government on many aspects related to connected intelligent technologies and data sciences.

Publisher Copyright:
© 2021

Keywords

Deep reinforcement learning
Open-RAN
Resource assignment
Self-play

Access to Document

10.1016/j.comnet.2021.108682Licence: Unspecified

Full-text PDF (submitted manuscript)Submitted manuscript, 907 KB

https://arxiv.org/abs/2103.02649

Handle.net

Persistent link

Cite this

@article{082b4ce4068b46feb8f560b1d942f5b3,

title = "Self-play learning strategies for resource assignment in Open-RAN networks",

abstract = "Open Radio Access Network (ORAN) is being developed with an aim to democratise access and lower the cost of future mobile data networks, supporting network services with various QoS requirements, such as massive IoT and URLLC. In ORAN, network functionality is dis-aggregated into remote units (RUs), distributed units (DUs) and central units (CUs), which allows flexible software on Commercial-Off-The-Shelf (COTS) deployments. Furthermore, the mapping of variable RU requirements to local mobile edge computing centres for future centralised processing would significantly reduce the power consumption in cellular networks. In this paper, we study the RU–DU resource assignment problem in an ORAN system, modelled as a 2D bin packing problem. A deep reinforcement learning-based self-play approach is proposed to achieve efficient RU–DU resource management, with AlphaGo Zero inspired neural Monte-Carlo Tree Search (MCTS). Experiments on representative 2D bin packing environment and real sites data show that the self-play learning strategy achieves intelligent RU–DU resource assignment for different network conditions. Comparing with baseline methods, including a heuristic virtual resource allocation algorithm, the Lego heuristic algorithm and the MCTS methods, the proposed approach achieves a performance gain between 5.70\% to 12.95\% in terms of resource utilisation efficiency.",

keywords = "Deep reinforcement learning, Open-RAN, Resource assignment, Self-play",

author = "Xiaoyang Wang and Thomas, \{Jonathan D.\} and Piechocki, \{Robert J.\} and Shipra Kapoor and Ra{\'u}l Santos-Rodr{\'i}guez and Arjun Parekh",

note = "Funding Information: This work is funded by the Next-Generation Converged Digital Infrastructure (NG-CDI) Project, supported by BT and Engineering and Physical Sciences Research Council (EPSRC), United Kingdom , Grant ref. EP/R004935/1 . Funding Information: Robert J. Piechocki is a Professor of Wireless Systems at the University of Bristol. His research expertise is in the areas of Connected Intelligent Systems, Wireless \& Self-Learning Networks, Information and Communication Theory, Statistics and AI. Rob has published over 200 papers in peer-reviewed international journals and conferences and holds 13 patents in these areas. He leads wireless connectivity and sensing research activities for the IRC SPHERE project (winner of 2016 World Technology Award). He collaborates on research grants totalling over £25M, and is a PI/CI for several high-profile projects in networks and AI funded by the industry, EU, Innovate UK and EPSRC such as NG-CDI, OPERA, AIMM, FLOURISH. He regularly advises the industry and the Government on many aspects related to connected intelligent technologies and data sciences. Publisher Copyright: {\textcopyright} 2021",

year = "2022",

month = apr,

day = "7",

doi = "10.1016/j.comnet.2021.108682",

language = "English",

volume = "206",

journal = "Computer Networks",

issn = "1389-1286",

publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Self-play learning strategies for resource assignment in Open-RAN networks

AU - Wang, Xiaoyang

AU - Thomas, Jonathan D.

AU - Piechocki, Robert J.

AU - Kapoor, Shipra

AU - Santos-Rodríguez, Raúl

AU - Parekh, Arjun

N1 - Funding Information: This work is funded by the Next-Generation Converged Digital Infrastructure (NG-CDI) Project, supported by BT and Engineering and Physical Sciences Research Council (EPSRC), United Kingdom , Grant ref. EP/R004935/1 . Funding Information: Robert J. Piechocki is a Professor of Wireless Systems at the University of Bristol. His research expertise is in the areas of Connected Intelligent Systems, Wireless & Self-Learning Networks, Information and Communication Theory, Statistics and AI. Rob has published over 200 papers in peer-reviewed international journals and conferences and holds 13 patents in these areas. He leads wireless connectivity and sensing research activities for the IRC SPHERE project (winner of 2016 World Technology Award). He collaborates on research grants totalling over £25M, and is a PI/CI for several high-profile projects in networks and AI funded by the industry, EU, Innovate UK and EPSRC such as NG-CDI, OPERA, AIMM, FLOURISH. He regularly advises the industry and the Government on many aspects related to connected intelligent technologies and data sciences. Publisher Copyright: © 2021

PY - 2022/4/7

Y1 - 2022/4/7

N2 - Open Radio Access Network (ORAN) is being developed with an aim to democratise access and lower the cost of future mobile data networks, supporting network services with various QoS requirements, such as massive IoT and URLLC. In ORAN, network functionality is dis-aggregated into remote units (RUs), distributed units (DUs) and central units (CUs), which allows flexible software on Commercial-Off-The-Shelf (COTS) deployments. Furthermore, the mapping of variable RU requirements to local mobile edge computing centres for future centralised processing would significantly reduce the power consumption in cellular networks. In this paper, we study the RU–DU resource assignment problem in an ORAN system, modelled as a 2D bin packing problem. A deep reinforcement learning-based self-play approach is proposed to achieve efficient RU–DU resource management, with AlphaGo Zero inspired neural Monte-Carlo Tree Search (MCTS). Experiments on representative 2D bin packing environment and real sites data show that the self-play learning strategy achieves intelligent RU–DU resource assignment for different network conditions. Comparing with baseline methods, including a heuristic virtual resource allocation algorithm, the Lego heuristic algorithm and the MCTS methods, the proposed approach achieves a performance gain between 5.70% to 12.95% in terms of resource utilisation efficiency.

AB - Open Radio Access Network (ORAN) is being developed with an aim to democratise access and lower the cost of future mobile data networks, supporting network services with various QoS requirements, such as massive IoT and URLLC. In ORAN, network functionality is dis-aggregated into remote units (RUs), distributed units (DUs) and central units (CUs), which allows flexible software on Commercial-Off-The-Shelf (COTS) deployments. Furthermore, the mapping of variable RU requirements to local mobile edge computing centres for future centralised processing would significantly reduce the power consumption in cellular networks. In this paper, we study the RU–DU resource assignment problem in an ORAN system, modelled as a 2D bin packing problem. A deep reinforcement learning-based self-play approach is proposed to achieve efficient RU–DU resource management, with AlphaGo Zero inspired neural Monte-Carlo Tree Search (MCTS). Experiments on representative 2D bin packing environment and real sites data show that the self-play learning strategy achieves intelligent RU–DU resource assignment for different network conditions. Comparing with baseline methods, including a heuristic virtual resource allocation algorithm, the Lego heuristic algorithm and the MCTS methods, the proposed approach achieves a performance gain between 5.70% to 12.95% in terms of resource utilisation efficiency.

KW - Deep reinforcement learning

KW - Open-RAN

KW - Resource assignment

KW - Self-play

UR - https://www.scopus.com/pages/publications/85123581219

U2 - 10.1016/j.comnet.2021.108682

DO - 10.1016/j.comnet.2021.108682

M3 - Article (Academic Journal)

SN - 1389-1286

VL - 206

JO - Computer Networks

JF - Computer Networks

M1 - 108682

ER -

Self-play learning strategies for resource assignment in Open-RAN networks

Abstract

Bibliographical note

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this