The way governments manage resources through the budget cycle has important implications for health policy and whether governments achieve societal objectives such as efficiency, equity, quality, and accountability. Studies found a positive association between health service delivery outcomes and good governance of public finance; however, the mechanisms through which public financial management affects service delivery remain underexplored. This article maps the three stages of the budget cycle to common performance criteria used in health service delivery. It applies this approach to experiences in Tanzania and Zambia. The findings point to a number of stumbling blocks, including the lack of flexibility to provide additional resources for unexpected demand for care, misalignment between budgeting and planning, fragmented funding sources, rigid internal controls, insufficient budget provision leading to arrears, and a budget evaluation system that is excessively compliance driven and gives inadequate attention to issues of equity, quality, and efficiency in service delivery.