...Cry and you cry alone.
So, until recently I have to admit I didn't find the Joker that frightening. He was more of a fun villain to me than anything else. This may be because of how he was played in the show (which is how I was introduced to the mythos). There are a few exceptions to this of course, and the one that has always creeped me out has been this running thing that the Joker will force people to smile or laugh. (Doesn't bother me in live action for some reason, just cartoons.) The one that always creeped me out the most was the instance of him doing this to Robin in the Batman Beyond film "Return of the Joker"...maybe it's because Robin is so clearly forced to do this that it plays more to me like child abuse and pedophillia than it does cartoon villainy. (There is a part where Robin is laugh-crying that always gets me.)
So there you go. It's not the Joker that gets me...it's crossing that safety line I think. I mean, it's one thing to have a villain that's embodying chaos by trying to orchestrate some insane plan to make people act crazy, or take a hit at his arch-nemesis...it's another when you have a villain that embodies chaos by screwing around with something innocent just because he can.