Some researchers claim that diverse populations lead to less trust. Generalized trust is a core value that leads to positive outcomes in societies--from greater tolerance of minority groups and immigrants and willingness to do good deeds, to less corruption, more social welfare and education spending, more open markets, and better functioning government. Generalized trust fundamentally rests upon a foundation of respect for diversity, but at the same time arguing that societies have a common culture. It is the idea that people have a shared fate. Generalized trust rests upon a foundation of economic equality. Yet some claim that diversity leads to less trust rather than more trust. Trusting people who are different from yourself is atypical of most people, they claim. I dispute this--arguing that generalized trust is largely unrelated to population diversity. It is not diversity that matters--it is how populations are distributed. I show that trust is lower not in diverse societies, but rather in societies with large minority groups that are segregated from the majority groups. Minority residential segregation leads to less trust because it leads to less interaction across different groups in society--and leads minorities to associate only with each other, to form their own political organizations, and to see their fate as less dependent upon majority groups. I then discuss how economic inequality and the rule of law shape the relationship between trust and minority residential segregation.