Existing studies have focused on the negative impact of inefficient resource allocation on energy performance in China’s factor market, but neglected to further explore the underlying reason for this phenomenon from the perspective of market segmentation. In this paper, the epsilon-based measure model, which combines the merits of radial and non-radial Data Envelopment Analysis, is employed to measure the energy efficiency, and price index method derived from Iceberg Transport Cost model is used to examine the degrees of market segmentation. On the basis, we use the Tobit model to empirically investigate the impact of market segmentation on China’s energy efficiency. The results show that although energy efficiency in the eastern region is higher than that in the central and western regions, the energy efficiency gap is narrowing significantly between the eastern and central, but insignificantly between the western and eastern. Although efforts have been made towards a unified national market, the western provinces still have more segmented markets than the eastern still. Econometric analysis indicates that market segmentation is negative to China’s energy efficiency significantly. This finding remains robust even if the endogeneity is excluded and the dependent variable is re-measured by the slack-based measure model, but is of a regional heterogeneity. We also find that factor market distortion, enterprises’ R&D investment, and industrial agglomeration are three mediation mechanisms through which market segmentation affects energy efficiency. In-depth analysis indicates that there is a Race to the Top competition centering on market segmentation among Chinese local officials in geospatial and economic space, which triggers a long-term inhibition to energy efficiency.