Absolutely!
I've done it myself (several times) as well as millions of people around the world. In fact, I prefer it that way as then you can really personally bond with your new baby. Those first few days are so special and precious. I loved being in a little bubble with just me, my husband, and the baby. As the husband, your job will be---do WHATEVER your wife asks you to do and tell her how wonderful she is at every turn.