I think we still are in RSV season, which is a time when brand-new babies are at special risk:
If I were you (and I have been, twice!), I'd stay home and get friends and family to come to YOU, bring you gifties, take care of you!
In some cultures, the baby doesn't leave the house for six weeks or more. If you're feeling housebound already, maybe some techniques to help you feel happier at home -- maybe do a little crafting? Work on the baby book? Surf the 'net? Cook something yummy? Eat bon bons and watch soaps?
Hell, all I wanted to do was SLEEP when #2 came along! And with baby #1 I was recovering from a c/section, temporarily living with my in-laws for six weeks so I wouldn't have to climb stairs ... oh yeah, now I remember cabin fever!
Anyway, if you take the baby out, try to wear him in a sling so people are less likely to touch him or want to pick him up -- when he's so tiny, he really is at risk for colds, and people in public places are veritable germfests. If anyone wants to touch him, tell them your doctor insists they go wash their hands first (that'll keep them away).
If he cries, nurse him! Chances are, that's what he'll want, either for food/drink or for emotional closeness. Remember, everything in the universe outside of your uterus is new to him! Some babies get overwhelmed easily. He can't even see very well yet -- it's best to keep him bundled close to you, anyway, so he feels safe and warm.
As for gear, that's been well-answered here already. I'll just add, give yourself time at home to get used to nursing comfortably and changing diapers so it won't seem like such a huge deal to do it elsewhere. (Do you have a sling, and have you figured out how to nurse him in it? That's a GREAT thing to own, if you're feeling at all shy about nursing in public!) Once you're accustomed to changing him on the changing table, practice changing him next to you on the couch, on the kitchen table (make sure you put down a big blanket or something so you don't leak poo where you eat!), on the floor ... when you do go out eventually, you don't know where you might end up having to change him. Balancing him on your lap as you sit on the toilet is not an unknown possibility! I'm not trying to scare you, just warn you to be prepared!
Oh, and P.S. CONGRATULATIONS! You're a Mom now! Enjoy your babymoon! What's your baby's name? Is he impossibly adorable?