This song is widely misinterpreted, which generated a great deal of exposure for the song and controversy for Newman. On the surface, the song is making fun of little people, but "short" is meant in a figurative sense, intending to poke fun at people who are short-tempered and small-minded, which is quite the opposite of the literal meaning. A lot of people didn't get the joke and thought of Newman as a bigot. Many radio stations refused to play it.
As always with Newman, if the song makes you feel like you need to sing along, you can be sure you're gonna feel bad about yourself after singing those lyrics. Still, at least you might learn something, since these are unusually insightful lyrics about prejudice: the way the whole song suggests that people tend to mix up aesthetic and moral failings ("short people are evil because they're ugly," or is it the other way around?), and the seemingly out-of-place bridge that suggests you can hold liberal platitudes in your head ("we're all brothers and stuff, man") while still being a nasty bigot. In short, this is the best song ever recorded about confirmation bias. Once you've decided you hate some type, everything about them just adds to your impression of their awfulness. Smart stuff, and still fun as all get out.