Great input so far.
For me, the biggest challenge to hunting one deer is OVER hunting. As JBird22 pointed out, you've got to be very disciplined, because focusing on one deer definitely increases the chances of burning the spot out and not only changing the buck's habits but even running him off! In general, mature bucks emphasize solitude/safety over all things. So simply being in the area more than a couple times dramatically increases the odds that the old guy will get fed up and move.
The buck I'm referring to lives in a remote spot that's very difficult to get to. I know my best chance of getting him is to go into this spot when conditions are perfect, and like dzsmith mentions, odds are best on the first 1-2 sits. After that everything can change. Also, my chances are BY FAR the best during archery season. Come rifle season, not to mention when the duck hunters converge on these watery places, it's just a crap shoot.