Do you know how many calories you're burning a day? Cutting 500 a day assumes you will lose a pound a week only if you're eating at a level that's maintaining your current weight.
3500 calories a day sounds like a lot to be maintaining a weight of 160, with three times a week exercise.
Said differently, if you cut back to, say 3000 or 2500 calories a day, and you're only burning 2200 a day, you are still going to be gaining weight, not losing it. High fiber foods will fill you up, and protein and healthy fat will keep you satisfied longer.
Cutting back gradually is definitely the way to go if you're worried about giving up and trashing your diet, but run the numbers first. FitDay does it, but I also like this calories burned in a day calculator for a "second opinion." http://www.freedieting.com/tools/calorie_calculator.htm