What family of Radial Basis Functions to use in Direct Policy Search? A comparative analysis

Direct policy search (DPS) is increasingly being used to flexibly design policies for multi-objective reservoir control. DPS is a promising approach that can easily find policies for many heterogenous objective functions, particularly when coupling global approximators with evolutionary algorithms. Nonetheless, specifying the topology and the family of global approximators, i.e., radial basis functions, is usually done by trial and error for practical applications and is often not reported in the literature. How does the selected family of radial basis functions affect the quality of the resulting control policies? Does the chosen family influence the search behavior of the evolutionary algorithms? Can we formulate recommendations for which families are more or less suitable in general, or given the characteristics of the control problem? We test a suite of radial basis functions to address these questions for finding Pareto optimal reservoir control policies using an established reference case. This reference case is the Conowingo reservoir, a transboundary water body in the Susquehanna River Basin in North-East US. The reservoir needs to meet multiple competing water needs for hydropower production, environmental flows, recreation, cooling water for Peach Bottom atomic power plant, and urban water supply for Baltimore, MD, and Chester, PA. To optimize the Pareto optimal reservoir control policies, we use the e-NSGA2 algorithm. Our study shows the effect of using different families of radial basis functions, particularly their impact on the recommended reservoir operations, the resulting tradeoffs across the different sectors, and the search behavior of the evolutionary algorithm.