At the risk of sounding wishy-washy, I put a 100 amp sub panel in my garage. All time and money decisions. So if you can swing it, I'll agree with Kenny even though he disagrees with me :crazy:.
Verify with local electrician, but I believe you are allowed to use #4 wire for that 100 amp sub panel for residential use. It could save you some money depending on your wire source. Continuing with my wishy-washy theme here, I used #2 wire because it was cheaper to buy that at Lowes than buy the #4 wire (when getting 500').
If you do switch things around (by using tandem breakers), I'd put low typical load stuff on the tandem breakers.
Pete