In a desktop grid computing environment, voluntary desktops (i.e., resource providers) are free to leave and join independently in the middle of execution. To develop a reliable desktop grid computing system, a scheduling scheme must consider the dynamic nature (i.e., volatility) of volunteers. Existing desktop grid computing systems, however, do not consider volatility in their scheduling procedures. As a result, job execution is often suspended, resulting in delayed completion time and degraded performance and reliability. To solve these limitations, we propose the Markov Job Scheduler based on Availability (MJSA) supporting three advanced scheduling schemes: OPTIMIST, PESSIMIST, and REALIST. These scheduling schemes are based on stochastic modeling of desktop availability. In the OPTIMIST scheme, in which time constraints are relaxed, the MJSA provides reliable resource selection at low cost. In the PESSIMIST scheme, where time constraints are rigid, the MJSA enables stable makespan in strictly time. Finally, in the REALIST scheme, where time constraints are only partially relaxed, the MJSA provides enhanced cost efficiency. In conclusion, the MJSA improves performance and reliability by adapting the appropriate scheduling scheme when selecting volunteers according to the needs of applications.