I really encourage you to put in a 200 amp panel. 100 amps sounds like a lot, but it is only 50 amps per leg. That isn't much.
First, it quickly adds up in lights and other high draw "background" equipment ( compressors, vacuum blowers, etc.). When you add the many low draw things you never think about, like a clock, radio, cell phone charger, computer, etc. It gets even higher. All this comes off the top before you switch on any piece of equipment.
Second, it doesn't take much to get up there with a HT oven going, a blown forge running, an angle grinder, a drill press, a sander, a mill, etc.
Third, even if you never plan on running more than 100 amps at a time, that is 50 amps per leg. You would have to carefully balance the equipment running to assure that you don't overload one side of the panel and trip the main breaker.
Fourth, wiring all those wall boxes and other circuits with 20-30 amp breakers will add up to three or four times the main breaker capacity. This is bad electrical wiring. I forget the code for how many amps of breakers you should put on a leg, but I suspect you would seriously go over it with a 100 amp box.