Illusion gets way better once you hit the perks that matter, but you're right that the early-to-mid stretch is rough. The issue is that by level 25-30 on anything above Adept, most humanoid enemies have high enough magic resist that your spells just bounce off, and the AI breaks out of crowd control too fast to matter.
Here's what actually saves the tree: the perk that makes spells stick longer on targets, and the one that makes your illusion spells affect higher-level enemies. Those two together turn Fury/Mayhem from a two-second gimmick into something that actually controls a room. You also want the quiet casting perk because casting an obvious spell in combat gets you rushed immediately.
The real play though? Stop trying to solo-carry with Illusion. Pair it with something else - Conjuration is the obvious choice since summoned stuff buys you time while enemies are confused, but even Destruction works. Cast your crowd control, let your summon or your damage spell do the heavy lifting, and suddenly Illusion feels way less useless. It's a support school pretending to be a primary damage dealer, and that's where it shines.
On Switch you're also stuck with vanilla scaling, which is harsher than SE/AE for magic builds in general. If you really want to make pure Illusion work, you might just have to accept it doesn't peak until later levels. But honestly? Conjuration branch is probably less frustrating for what you're describing.
