For decades, scholars and politicians have been engaging in the quest for an institutional model that prevents conflict and maintains peace in ethnically divided societies. Many propose a federal system, yet with different intentions: Segregationists argue that giving territorial autonomy to the different groups by creating homogeneous federal units will mediate political conflict and prevent secession by reducing discrimination of minorities. Integrationists reject this idea, claiming that integration in form of heterogeneous federal units promotes tolerance and thereby peace. This study sheds light on the debate by comparing India’s federal states, which differ in their levels of religious heterogeneity. The results of the quantitative analysis indicate that religiously homogeneous federal units have higher levels of interreligious tolerance than heterogeneous units, lending support to the segregation thesis. However, the occurrence of communal riots in the different states does not seem to be affected by their degree of heterogeneity.