Unarmed at level 12 is rough, yeah, but it does get way better - you're just hitting the wall where the game hasn't scaled to your build yet. The damage feels terrible right now because you're missing the perks that actually matter.
What you need is the perk that lets your unarmed strikes ignore a chunk of armor, and then the ones that add flat damage per hit. There's also one that staggers enemies on power attacks. Once you stack those together with your enchantments, you'll stop feeling like you're punching a wall. Problem is most of those are deep in the tree, so yeah, level 12 you're still in the slog.
The other thing - and this might sound weird coming from someone who loves lore - but unarmed is kind of a late-game build in vanilla Skyrim. It's not that it's broken or anything, but the game was designed around weapons and magic first. If you're on Switch you can't mod it, so you're stuck with what Bethesda gave you. That said, if you're actually invested in it, push through to maybe level 25-30 and grab the key perks. It clicks way harder once you do.
Alternatively, if you want to keep the hand-to-hand fantasy but make it viable sooner, heavy armor plus unarmed is legitimately good - you tank everything and just wear enemies down. Less pure, but honestly more fun to actually play.