The Markov chain model (MCM) has become a popular tool in the agricultural economics literature to describe how farms experience structural change and to study the impact of various drivers of this process, including public support. Even though some studies have accounted for heterogeneity across farms by letting transition probabilities depend on co- variates depicting farms and/or farmers' characteristics, only observed heterogeneity has been considered so far. Assuming that structural change may also relate to unobserved farms and/or farmers' characteristics, we present how to implement the mover-stayer model (MSM) which considers a mixture of two types of farms, the `stayers' who always remain in their initial size category and the `movers' who follow a first-order Markovian process, and how to estimate it thanks to the expectation-maximization (EM) algorithm. This modeling framework relaxes the assumption of homogeneity in the transition process which grounds the usual MCM. An empirical application to a panel of French farms over 2000-2013 shows that the MSM outperforms the MCM in recovering the underlying year-to-year transition process as well as in deriving the long-run transition matrix and forecasting future farm size distributions.