Field boundaries with permanent vegetation cover are a key habitat for farmland biodiversity. Buffer zones are wide field boundaries established to prevent nutrient leaching and erosion into waterways and serve as habitats for farmland wildlife. Our main hypothesis was that several years of grazing or cutting management results in greater plant species richness and heterogeneity in buffer zones in comparison with sites managed for only a few years. We also hypothesized that litter cover and soil phosphorus (P) level would decrease after several years of management, in comparison sites with relatively few years of management. Through this study we aimed to gain a better understanding of how to increase biodiversity in buffer zones. The study included 15 buffer zones within a single landscape. Mean species richness was significantly higher in the group of sites grazed over several years (21 species) in contrast to extensively cut sites (14.8 species) and/or grazed sites (18 species). Species heterogeneity did not respond to different management regimes. Management, slope aspect and litter were significant explanatory factors for species composition. The amount of soil phosphorus measured at three different depths was significantly lower in the buffer zones managed by cutting or grazing for a few years in contrast to those that were grazed for several years. Management that positively affected species diversity did not result in the expected decrease in soil phosphorus. Therefore, we propose that the greater species richness at grazed sites results mainly from disturbances caused by grazing.