Well, I'm speaking as someone who has recently started writing fiction myself (especially horror). I finished a story this afternoon that I found very disturbing, because it depicted acts of abuse and violence I am uncomfortable with. I wanted to cut parts from the story, but then there would be no story. Particularly the verbal abuse, I hated writing it, but it was exactly what made it frightening. I do think about the limits of taste. I will admit I could not finish Full Dark, No Stars because one of the stories pushed that limit.
A story I read recently by Joe Hill really gets into this subject very well, called Best New Horror. It's a disturbing story, it sticks with you whether you want it to or not, but it's a disturbing story about disturbing fiction. For me, my answer is if it's going to be graphic, it needs to be graphic for a reason. It can be brutal, but doesn't have to be gratuitous. I'm torn on Best New Horror, because it feels like it went so far, but I still think about the implications. It worked at making me think, and it had to shock me to do it.
There is no rating or limits to fiction, no line you can draw to say what is and is not appropriate. There are standards of taste and decency that change (look at the It example). There are some people who are numbed to shock, so satiated that they wouldn't care how graphic your story is. They'd crave the brutality and obscenity for its own sake. People aren't all shocked by the same things.
What I think you should consider is why you are writing the subject matter. How it affects the story. Even if it's just like you say, that it's untouched. I do have an idea of my limits, what I will and won't do, what I will and won't read, and I would say it's healthier to keep away from darkness when you can. If the fiction is damaging, to you or to readers, then you have an answer. That's the limit.
And I do agree that horror is found outside of graveyards. Which is why of my stories so far, only one has had any supernatural elements, and even still it was about acts of people. The evil in humanity is much scarier than an invisible ghost.