Deep reinforcement learning with spatial and temporal awareness for active boundary control of buoyancy-driven convection

Engineering notes

Engineering notes will be added by the Power for Solar editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为光伏效率、钙钛矿太阳能电池、储能技术、太阳能热利用、BIPV、并网技术等高价值论文补充中文说明。

Original abstract

Deep reinforcement learning (DRL) applied to thermal convection control consistently produces \textit{degenerate actuation}: wall-temperature policies whose outputs are saturated, pseudo-random, or spatially incoherent. Two compounding deficiencies are responsible: multilayer-perceptron policies that discard spatial flow structure, and memoryless policies that cannot distinguish self-induced flow changes from background evolution. Together they prevent the discovery of physically meaningful control laws even when cell coalescence (the merging of convection rolls into fewer, larger structures), which would reduce $\mathrm{Nu}$, is accessible to boundary actuation. The present framework addresses both causes through four targeted design choices: convolutional policy networks, Gated Recurrent Unit (GRU) memory, off-policy training (TD3/MADDPG), and action-smoothness constraints. A systematic $2\times2$ factorial design isolates the contribution of each component. On Rayleigh--Bénard convection at $\mathrm{Ra}=10{,}000$, all four configurations achieve cell coalescence and reduce $\mathrm{Nu}$ to as low as $1.83$ ($26\%$ below the uncontrolled baseline) in 350 episodes, without the full-field data augmentation required by prior work. Crucially, coalescence is achieved even by the single-agent configuration, demonstrating that the multi-agent formulation is not a prerequisite once the policy architecture is sufficiently expressive. Applied to double-diffusive convection in the salt-finger regime, the framework spontaneously discovers a travelling-wave actuation whose phase speed adapts to the evolving mixing state of the flow, enhancing heat transfer by $19.1\%$ and reducing salinity variance by $21.0\%$.

Links and sources

• Official / arXiv page
• PDF from original source

Comments