Climate change impacts on agricultural production will shape the challenges of reaching food security and reducing poverty across households in the future. Existing literature lacks analysis of these impacts on different household groups under consideration of changing socio-economic developments. Here, we analyze how crop yield shifts induced by climate change will affect different household types in three low and low middle-income countries, namely Vietnam, Ethiopia and Bolivia. The long-run analysis is based on a recursive-dynamic Computable General Equilibrium model. We first construct a baseline scenario projecting global socio-economic developments up to 2050. From there, we implement business-as-usual climate change shocks on crop yields. In the baseline, all households benefit from welfare increases over time. Adding climate change induced yield changes reveals impacts different in size and direction depending on the level of the households’ income and on the share of income generated in agriculture. We find that the composition of the factor income is of large importance for the vulnerability of households to climate change, as, the loss for non-agricultural households is highest in absolute terms. The complementary comparative static analysis shows smaller absolute and relative effects for most households as the differentiated factor income growth over time is not considered, which makes household types more or less vulnerable. A sensitivity analysis varying the severity of climate change impacts on yields confirms that more negative yield shifts exacerbate the situation of the most vulnerable households. Furthermore, it underlines that yield shocks on staple crops are of major importance for the welfare effect. Our findings reveal the need for differentiated interventions to mitigate consequences especially for the most vulnerable households.