EGU2020-17461, updated on 12 Jun 2020
EGU General Assembly 2020
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Fitted Q-iteration for optimal water reservoir network operation under varying hydro-climatic conditions

Bin Liang1,2, Matteo Giuliani1, Liping Zhang2, Senlin Chen2, and Andrea Castelletti1
Bin Liang et al.
  • 1Department of Electronics Information and Bioengineering, Politecnico di Milano, Milan, Italy (
  • 2Shcool of Water Resources and Hydropower Engineering, Wuhan University ,Wuhan, China (

Although being one of the most important approaches to design optimal water reservoir operating policies, the Stochastic Dynamic Programming is challenged by the three curses of dimensionality, modeling, and multiple objectives that make it unsuitable in most practical applications. Increased hydrological variability induced by climate change and human activities further challenges the control of hydraulic infrastructures calling for more flexible and efficient approaches to operation design. Tree-based fitted Q-iteration (FQI) is a value-based, offline and batch mode reinforcement learning method, which employs the principles of continuous approximation of value function through non parametric randomized ensemble of regression tree, i.e. Extremely Randomized Tree. So far FQI has been used for relatively simple systems, including one dam and several state variables, and looking at historical hydrology. In this work, we explore the potential for FQI to design reservoir network operation under varying hydro-climatological conditions. The approach is demonstrated on a real-world case study concerning the optimal operation of a network of three water reservoirs in the Qingjiang River basin, China. Preliminary results show that the computational efficiency and performance of the policies derived by FQI are all satisfactory compare to traditional Stochastic Dynamic Programming, and the advantages in terms of computational efficiency and policies performance become more relevant when evaluated considering uncertain hydro-climatological and socio-economic conditions that requires using more information for conditioning the control policy. 

How to cite: Liang, B., Giuliani, M., Zhang, L., Chen, S., and Castelletti, A.: Fitted Q-iteration for optimal water reservoir network operation under varying hydro-climatic conditions, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-17461,, 2020