A concentration of poor in rural areas has resulted in a research and policy focus on agricultural technologies and (poor) households' impact on soil productivity. But farm households do not live of farming alone, non-farm activities play a principal role even in remote areas. With a unique household-level dataset covering seven regions in Africa and two in Asia we analyze (1) the importance of non-farm income in different geographical zones; (2) the role of geographical factors in determining access to non-farm employment; (3) the role of non-farm income in external input use and soil nitrogen balances. Distinguishing geographical zones based on the distance to urban areas we find the share of non-farm income increasing from 12 percent in the remote areas to 35 percent in peri-urban areas. Geographical location is found to explain a major part of the variation in individual non-farm participation, besides characteristics like education and gender. At household level we find non-farm income not playing a role of significance in explaining external input use, inorganic fertilizer use nor changes in the nitrogen balance. Households thus appear not to invest non-farm income in agriculture. This limits the contribution of non-farm income to reducing widespread soil nutrient depletion witnessed in Africa.