This paper shows that econometric models that include categorical variables are not invariant to choice of ‘base’ category when random parameters are estimated, unless they are allowed to be correlated. We show that the lack of invariance can lead to significant increases in Type I errors, and a misrepresentation of the preferences of respondents. We hypothesis that these biases may influence the economic policy implications of published models that contain this error, which we show in two empirical applications. However, it is impossible to identify the degree of the error in the many published papers we identify that contain this effect, without re-estimating the models correctly.