The combat itself isn't clunky, but it feels fundamentally different because of how the math works underneath. In Skyrim you're mostly just... swinging and watching damage numbers. Oblivion's attribute system means your actual hit chance, damage output, and how much you get hit for all scale with stats that level up based on what you do, not just your character level. So if you're a sword user but you've been casting spells a lot, your sword damage won't scale as fast as it could.
The real adjustment is that you can't just bumble through on difficulty like you might in Skyrim. If you level poorly (which is super easy to do early on), you'll hit a wall where enemies scale faster than your damage does. It's not frustrating once you understand it, but yeah, your first playthrough might feel rough around level 10-15 if you're not paying attention to which skills you're actually using in combat.
My advice: pick a weapon type and stick with it for a while. Don't level Blade and Blunt equally. Same with magic - if you're going to cast, commit to one or two schools. The game rewards specialization way more than Skyrim does. Also, don't sleep before leveling up your Endurance if you can help it, since that's the only stat that doesn't retroactively apply to your health pool.
It's a different rhythm but honestly once it clicks it's great. Way more tactical than Skyrim's combat.