Ulfric also segregates the dark elves in his city. The first step in what will eventually become imprisonment, then either exile or extermination. In order for the Stormcloaks to be successful, he has to plant the idea of racial superiority. At first, observers might think it is just "national or racial pride", but if the Stormcloaks are successful in driving out the Legion and establishing power in Skyrim, the next step would be racial purification. The same thing happened in Germany. It's all in history. The Nazi party segregated and robbed the jewish people, then once the Nazi's became more powerful they imprisoned them because they weren't Aryan. We all know what happened next.
I'm not saying Ulfric would do the exact same thing as the Nazi's, just that the similarities are undeniable in what the Nazis did with the SA, then later the SS.