You are correct; it's a process. Also correct that you don't build 2 pounds of muscle in a week (a lot of people make the mistake of thinking that a gain or lack of loss is muscle, when in reality, it does not build that fast or that much for most of us). However, I feel strongly that the culprit is the TOM. I reliably gain 3-5 pounds every time, and judging from the boards over the past two years, it happens to most if not all of us to a degree. So be patient, do what you know how to do, and put the scale away for a week if you have to!
And now that you don't have to be perfect, you can be good.