Back extensions:
Back Extensions
or with a ball:
Back Exercises - Back Extensions on the Ball
Don't forget your latissimus dorsi (I think I spelled that right...?):
Back Exercises - Strength Training Exercises for the Lats and Lower Back
Dumb bell rows:
Dumbbell Rows
A strong core supports your every-day functions, improves your posture and impacts how you carry yourself. You'll have more stamina for things like long sessions at a desk, too.