Since 50 years, the Common Agricultural Policy (CAP) has impacted the evolution of agricultural landscapes by driving changes in land use and farming practices in Europe. Based on a critical literature review, this paper discusses to what extent the CAP is contributing to the management of EU agricultural landscapes. Agricultural landscapes are described as a multi-scale public good. Optimal management of agricultural landscapes requires actions on three scales: (1) the management of landscape elements at farm level, (2) the integration of farms in the agricultural landscape at landscape level, and finally (3) the conservation of the diversity of agricultural landscapes in EU as a global public good. The CAP has until now mainly focused on the first scale. We show how policy instruments could be refined for the CAP to integrate the two other scales. This paper provides a knowledge base to support an effective CAP policy design in the direction of improved landscape management, an important component of the EU project towards a more sustainable agriculture.