Well I grew the Aji Habanero variety a few years ago, it was labeled as a very mild in heat sweet pepper. I remember it being "very seedy" and not much sweetness to speak of. I think the amount of seeds and bitterness of them affected the taste of the pepper greatly.
I cant really say I dislike many peppers, as I think some are better in certain applications and such. I like Aji Peppers pickled like pepper rings, so in the right application all peppers are pretty good usually.
That being said the Aji Habanero was a real disappointment for me.