Using data from the Living Standards Measurement Surveys-Integrated Surveys of Agriculture (LSMS-ISA), this paper investigates the determinants of changes in youth and women participation in agriculture. Participation in the agricultural labor force is measured using hours per week in agriculture and change in hours worked per week in agriculture between two survey waves for Nigeria and Uganda. Ordinary Least Squares and Tobit methods are used to estimate the model. The findings suggest that age is a strong determinant in hours worked per week in agriculture in Nigeria but not in Uganda. For both countries, age does not seem to have an impact on changes in hours worked per week in agriculture by the youth or by women. Nigerian men work more hours per week in agriculture than women while the opposite is true for Uganda. Education, gender, rural residence, and non-agricultural wage income strongly affect hours worked per week in agriculture.