For a newcomer to paleo, you've pretty much nailed the short budget category grocery list!
There are a lot of resources on the web to help you combine all those tasty things in delicious ways. The Primal Blueprint website (marksdailyapple.com) itself has an extensive recipe page. If you search for "grok" and "recipes" I'm sure you could find tons more.
1) eat real food - more vegetables, moderate meat, moderate fruits, less grains, less sugar, less vegetable oils.
2) exercise - moderate intensity cardio, sprinting, heavy lifting, dedicated stretching and mobility.
3) live - relax, de-stress, meditate.
Disclaimer: I'm not professionally qualified to make any formal recommendations. I've just done my homework and I'm my own guinea pig. All of my data, unless otherwise cited, comes from a sample size of n=1 (me).