Too often deadlifts enter lumbar flexion, where your spine is curved like an inverted "J". This not only makes you look like a frightened cat, it's also an unsafe pulling position – especially when it occurs because of the spine changing position/shape under the load (rather than starting out that way).
Doing back extensions and reverse hypers can help your cause, but one way to get better at deadlifting is to continue practicing the actual movement, but with a slight modification.
Take a couple of plates off the bar and practice doing some paused reps. Freezing a few inches off the ground on the way up forces the spine to remain neutral and prevents the hips from shooting up first.
Likewise, the quads and glutes get a better chance to contribute to the lift, making the lower back receive the assistance it needs and deserves. Do 5 sets of 3-5 reps once a week. You can do them in addition to your regular deadlift workout.
While paused deadlifts can do wonders in shoring up your spinal position, they can also help with balance. There should be a slight backward emphasis (about 5 degrees) when picking up the bar to counterbalance the load that's in front of you, especially when it's heavy.
If you watch the full-deadlift video below closely and pause it on any given completed rep, you'll notice that slight backward emphasis.
Being completely vertical at the top of a deadlift probably means you let your back dominate the lift and forfeited some hip drive. Finishing up on a backward "slant" of about 5 degrees properly counters the load in your hands.
Remember, this isn't a trap bar with a weight distribution that's evenly distributed around the body. It's a barbell and all the weight is in front of you. Using the paused deadlift can help you sit back ever so slightly in order to stop the torso from lurching forward and throwing off your force curve.