A macro, which automatically executes a series of spreadsheet software commands, is developed to find the steady-state condition of a Markov decision process (MDP). The typical infrastructure management approach is to assess system conditions, then select and implement mitigating strategies with the joint objectives of maximizing benefits and minimizing costs. Infrastructure management decisions must often be made under conditions of uncertainty, making a probabilistic model of the system useful. An MDP is a relatively simple model that exhibits stochastic dependence — stochastic in that future conditions are based on probabilities, and "memoryless" dependence in that the next condition is based only on the existing state. One of the features of an MDP is the achievement of a steady state following repeated applications of a management policy. Toward facilitating the analytical requirements of an MDP, a computerized procedure to determine the steady-state condition of an MDP is developed. The procedure might serve as one component of a complete MDP-based infrastructure management system. A comprehensive system would also compute the rewards and costs associated with a given policy and determine the optimal policy, possibly through a linear programming formulation.
