A growing body of literature suggests that in utero exposure to hunger negatively affects children’s survival and linear growth. In this paper, we retrospectively linked data on local agricultural output and household food reserves during the in utero period to children’s health and nutritional status in the first five years of their life. We hypothesized that seasonal variations in agricultural yields and food reserves affect the quantity and diversity of food intake during pregnancy, and that pregnancies during periods with limited food reserves are associated with poorer child health outcomes. We generated a food reserve scarcity index (FRSI) based on reported food stocks at the household level reported in post-harvest surveys from 2001-2007 and estimated associations with child survival, birth size and World Health Organization (WHO) growth Z scores using multivariable regression model. We found negative and statistically significant associations between children’s weight and height Z-scores (WAZ and HAZ) and food scarcity in all trimesters with largest associations for the first and third trimesters. While we found that food scarcity in the second trimester increases children’s mortality risk, food scarcity in early gestation had protective effects on mortality. The results suggest that policies aimed at reducing vulnerability to food scarcity require targeting the vulnerable populations and proper timing of policies. Policy implications encompass two pathways: One is through nutrition such as food aid and supplements; And with the recurrence of food scarcity problem, the second more sustainable solution is through agriculture and extension such as proper food storage.