We address the problem of controlling two classes of Markov processes, viz., Markov chains and diffusion processes. Our main goal is to minimize a long-run average cost criterion (or ergodic cost criterion) associated with ...