Food away from home (FAFH) is known to be of poorer nutritional quality. Given the increased prominence of FAFH over the past 35 years, some policymakers and health advocates seek to further regulate this market. This policy approach is only as strong as its expected impact. Using an individual fixed effects approach, we first document how FAFH (both fast food and restaurants) has had a relatively stable negative impact on dietary quality over 1977-2010. Yet, overall levels of dietary quality have increased over 20% during this period. To this end, we use an Oaxaca-Blinder decomposition approach to understand how changes in FAFH choices and observable characteristics of individuals have impacted the trajectory of dietary improvements. We find that increased consumption of FAFH plays a very small role in explaining changes in dietary quality (about 4%) as compared to changes in demographics (e.g., age and race/ethnicity -- about 5%) and human capital (e.g., education -- about 7.5%). We attribute the substantial residual, or unexplained, improvement to changes in unobservable factors (e.g., consumer preferences, the food environment, and overlapping nutrition policy initiatives).