I make a real effort to slow down my backswing. If I jerk my driver back too quickly, then I have to think about stopping my momentum at the top. That prevents me from making a smooth transition from backswing to downswing exactly where my downswing plane begins to take shape. When I'm hitting it well I'm transitioning well, which allows me to smoothly accelerate from the top all the way to the back of the ball.
If you're slow going back and smooth at the top, you'll be fast where it counts: at the bottom.