Investigations into Deep Reinforcement Learning for Wind Farm Set-Point Optimisation

Wake steering is a form of wind farm flow control in which upstream turbines are deliberately yawed to misalign with the incoming wind in order to reduce the impact of wakes on downstream turbines. This technique can give a net increase in the power generated by an array of turbines compared to standard greedy control where each turbine acts for its own benefit by aligning with the incoming wind. However, optimising the set-points of multiple turbines under varying wind conditions can be prohibitively complex for traditional, white-box models. Reinforcement Learning (RL) agents learn optimal long-term behaviours through "trial-and-error", making them suited to controlling arrays of wind turbines under changing wind conditions for maximum farm power. Related works applying RL to this problem have tended to concentrate on either single wind directions or ranges up to around 10degs. Here, the Deep Deterministic Policy Gradient algorithm has been used to train RL agents to control a nine-turbine array to implement wake steering under multiple wind directions between +/- 45degs. While the agents were trained on steady-state (time-averaged) wind flow data, the performance of the final agent was tested on "quasi-dynamic" wind flow with varying wind direction. Under these conditions, the final agent achieved an average of 7% more power than greedy control per direction. This agent was then used to control the wind farm under a smaller subset of directions including many not seen during training, gaining on average 17% additional farm power per direction compared to greedy control.