In this article, we present a multistage model to optimize inventory control decisions under stochastic demand and continuous review. We first formulate the general problem for continuous stages and use a decomposition solution approach: since it is never optimal to let orders cross, the general problem can be broken into a set of single-unit subproblems that can be solved in a sequential fashion. These subproblems are optimal control problems for which a differential equation must be solved. This can be done easily by recursively identifying coefficients and performing a line search. The methodology is then extended to a discrete number of stages and allows us to compute the optimal solution in an efficient manner, with a competitive complexity. (c) 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 32-46, 2016