It's not just the not-very-subtle at all racism of his fiction, but the considerable collection of racist comments he made in correspondence as well. It's not an item of speculation -- it's an established fact, so it's hard to avoid.

As to sexism: this seems to be less well-perceived, largely because he did everything possible to render women completely invisible in his world.

On the political overtones I've mentioned: I haven't really seen much commentary on them.