We test and quantify the (in)stability of farmer risk preferences, accounting for both the instability across elicitation methods and the instability over time. We used repeated measurements (N=1530) with Swiss fruit and grapevine producers over 3 years, where different risk preference elicitation methods (domain-specific self-assessment and incentivized lotteries) were used. We find that farmers’ risk preferences change considerably when measured using different methods. For example, self-reported risk preference and findings from a Holt and Laury lottery correlate only weakly (correlation coefficients range from 0.06 to 0.23). Moreover, we find that risk preferences vary considerable over time too, i.e. applying the same elicitation method to the same farmer in a different point in time results in different risk preference estimates. Our results show self-reported risk preferences are moderately correlated (correlation coefficients range from 0.42 to 0.55) from one year to another. Finally, we find experiencing climate and pest related crop damages is associated with farmers becoming more risk loving.