Yeah, you've actually hit the real problem and it's not what you think. You said you've got all the One-Handed perks maxed — but you're using a bow. One-Handed doesn't affect Archery at all. You're only getting the base Archery perks, which at level 45 if you haven't been pumping that tree hard, leaves you doing way less damage than you think.
The armor thing is legit though. Heavy Armor scaling does fall off compared to enemy damage output around mid-game if you're not actively leveling it alongside your combat skills. At 45 you should have it pretty high, but if you've been neglecting it in favor of Archery, that's your glass cannon problem right there.
Quick fix: dump some perks into the Archery tree if you haven't already — the ones that boost damage per draw, critical damage, and the perk that lets you draw faster are huge. For armor, grab the perks that reduce incoming damage and boost armor effectiveness (the ones in the mid-tree). You don't need to reroll; stealth archer is totally viable at 45. You're just underinvested in the actual skills keeping you alive.
What's your Heavy Armor level sitting at right now?
