It's been said before, but I'll say it again. Revisions are crazy hard.
Not so much the grammatical stuff--that just takes patience.
Not even the editing of plot--thinking through the arc of your story and realizing parts of it are crap or could really be a whole lot better. Maybe it takes a while, but in the end it's easy. Because we know what's crap and what's pristine and glorious. Even if we push it to the back of our brains for awhile and convince ourselves otherwise, we always know when something isn't working. Eventually.
So, no, it's not that. The hardest part is sitting down in front of your computer or a printed manuscript, setting your hands on the keyboard or firmly clasped around a pen, and pressing "delete" when you still love what you've done. Scratching out half the words. Looking at your work and realizing that though it isn't bad (maybe it's even really good!), it doesn't fit within the context of the story. That it's not really necessary to the complications and resolution; it's just back-story.
I had to do this, recently. My love for the back-story had me ignoring how unnecessary it was to the actual plot, until a wonderful beta partner snapped me out of it. All 70 pages, gone. Vanished.
But I've realized that it's better this way. It wasn't that the back-story was bad--really, it was pretty well-written. It was just wasn't necessary. That doesn't mean parts of it never happened, because all characters have back-story. It just means a reader won't get to see it.
But that's okay. I would be happy sharing only pieces of this world in my head with readers. My novel, minus those 70-odd pages, is enough.