In fiscally decentralized countries, sub-national governments (SNGs) may face soft budget constraints and consequently invest and borrow too much. The policy literature claims that, with competitive capital markets and central governments imposing hard budget constraints (HBCs), inefficient investment by SNGs should not arise. We present a model where this is not the case: HBCs can be too "hard" and discourage investment that is socially efficient. The model combines a dynamic commitment problem as in Kornai, Maskin and Roland (2004) for central government with a moral hazard problem between central and SNG. The HBC over-incentivises the SNG to provide effort by penalizing it too much for project failure, thus leading ultimately to the possibility that socially efficient projects may not be undertaken.