Deriving optimal single-reservoir operating policies with reinforcement learning based approach incorporating uncertainties of demand and streamflow

Obtaining optimal reservoir operation policies is a challenging task and a strategic concern for policymakers. These policies are typically derived through a complex decision-making process with conflicting objectives, represented by nonlinear, nonconvex and multi-modal functions. The information on available inflow and various demands play a key role in developing optimal operation rules. However, they are characterized by various uncertainties which reduce the practical applicability of deterministic policy solutions. In literature, most of the studies handle streamflow uncertainty with single-demand scenarios. Although Stochastic Dynamic Programming (SDP) is a widely-used method for reservoir operations optimization under uncertainty, it suffers from the dual curses of dimensionality and modeling. This study considers the uncertainties for streamflow and various demands such as municipal, industrial, hydropower and irrigation water requirements. Here, we present a reinforcement learning framework that utilizes uncertainty-aware streamflow forecasts and demand requirements to yield optimal operation policies for Sardar Sarovar Dam, India. The proposed methodology incorporates the uncertainties of the underlying inflow and demand behavior, and demonstrates better performance than SDP in terms of net benefit. Overall, this work offers reliable techniques that can be used to develop multi-objective reservoir operation policies which are more adaptable in real-time.