State Departments of Transportation spend a considerable portion of their winter maintenance budgets getting accurate weather forecasts that are essential for proactive operations and are interested in getting the best value for their money. With that intention, the Kansas Department of Transportation obtained weather forecasts from three commercial weather data providers for four selected locations within the state of Kansas. This paper presents a quality analysis of those forecasts to identify the most accurate and reliable provider. In doing so, researchers used the prediction of frost events as the performance measure. Several different measures that were capable of quantifying the selected attributes (reliability, accuracy, and skill) were used to evaluate different forecasts for each location and for combined data.