We formulate the day-ahead bidding problem for a hydropower producer having several hydropower plants residing in a river basin. We present a novel approach inspired by Dynamic programming with approximations in value and policy space by neural networks. This allows for more accurate modeling of the problem by avoiding linear approximations of the production function and bidding. Stochastic programming is a method frequently used in literature to solve the hydropower production planning problem. Stochastic programming is used on linearized systems and under assumptions of known distributions of the involved stochastic processes. We test the proposed algorithm on a simplified system, suitable for Stochastic Programming and compare the obtained policy with the results from Stochastic Programming. The results show that the algorithm obtains a policy similar to that of Stochastic Programming.