This paper proposes a Markov decision process (MDP) based approach to derive an optimal schedule of maintenance, rehabilitation and replacement of the water main system. The scheduling problem utilizes auxiliary information of a pipe such as the current state, cost, and deterioration model. The objective function and detailed algorithm of dynamic programming are modified to solve the periodic replacement problem. The optimal policy evaluated by the proposed algorithm is compared to several existing policies via Monte Carlo simulations. The proposed decision framework provides a systematic way to obtain an optimal policy.