A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

Wei Wang; Peizheng Li; Angela Doufexi; Mark A Beach

doi:10.1109/LCOMM.2025.3568652

A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

Wei Wang^*, Peizheng Li, Angela Doufexi, Mark A Beach

^*Corresponding author for this work

Research output: Contribution to journal › Article (Academic Journal) › peer-review

8 Downloads (Pure)

Abstract

Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.

Original language	English
Pages (from-to)	1579-1583
Number of pages	5
Journal	IEEE Communications Letters
Volume	29
Issue number	7
Early online date	9 May 2025
DOIs	https://doi.org/10.1109/LCOMM.2025.3568652
Publication status	Published - 1 Jul 2025

Bibliographical note

Publisher Copyright:
© 1997-2012 IEEE.

Access to Document

10.1109/LCOMM.2025.3568652

Full-text PDF (accepted manuscript)
This is the accepted author manuscript (AAM) of the article which has been made Open Access under the University of Bristol's Scholarly Works Policy. The final published version (Version of Record) can be found on the publisher's website. The copyright of any third-party content, such as images, remains with the copyright holder.
Accepted author manuscript, 740 KBLicence: CC BY

Handle.net

Persistent link

Cite this

@article{6e79aa9ffdbf4ab38a4a632422e81f58,

title = "A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs",

abstract = "Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.",

author = "Wei Wang and Peizheng Li and Angela Doufexi and Beach, \{Mark A\}",

note = "Publisher Copyright: {\textcopyright} 1997-2012 IEEE.",

year = "2025",

month = jul,

day = "1",

doi = "10.1109/LCOMM.2025.3568652",

language = "English",

volume = "29",

pages = "1579--1583",

journal = "IEEE Communications Letters",

issn = "1089-7798",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

number = "7",

}

TY - JOUR

T1 - A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

AU - Wang, Wei

AU - Li, Peizheng

AU - Doufexi, Angela

AU - Beach, Mark A

PY - 2025/7/1

Y1 - 2025/7/1

N2 - Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.

AB - Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.

U2 - 10.1109/LCOMM.2025.3568652

DO - 10.1109/LCOMM.2025.3568652

M3 - Article (Academic Journal)

SN - 1089-7798

VL - 29

SP - 1579

EP - 1583

JO - IEEE Communications Letters

JF - IEEE Communications Letters

IS - 7

ER -

A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

Abstract

Bibliographical note

Access to Document

Handle.net

Fingerprint

Cite this