I try to keep it to two a day as well — I am also a fellow coffee-holic! If I didn't restrain myself I'd probably go a bit crazy on the coffee. Coffee is good for some things, like modestly increasing basal metabolic rate (which helps burn more fat at rest) and having some antioxidant and protective benefits for the liver, but having too much can harden your arteries in much the same way nicotine does. This is because chronic caffeine use raises cortisol (the stress hormone), and this in turn causes a bunch of other issues like interfering with sleep (very important to fat burning), raising blood pressure, and maybe even causing heart problems.
The easiest way to cut back for you would probably be to switch out the large coffees for regular or small. You can even start using cream or whole milk again because this will keep you fuller. But two small coffees is very doable, I have one in the morning and one between lunch and dinner.