We examine how changes in fiscal deficits affect near-term future inflation in a panel of emerging market and developing economies (EMDEs). Using a novel method for quantile panel regressions with fixed effects, we find that an increase in the fiscal deficit has highly non-linear effects on inflation - that is, a larger impact on upside tail risks than on average inflation. These effects are substantially larger in EMDEs than in advanced economies. We then show that an increase in the fiscal deficit raises the risk of future currency depreciation which magnifies the initial inflation response. This external channel is closely related to sovereign risk, being greater when the share of sovereign debt in foreign currency is large or when a sizeable share of sovereign debt is held by foreign residents. Finally, we find that the effects of fiscal deficits on future inflation are strongly attenuated in inflation targeting regimes and also influenced by constraints on monetary policy.