Just attempt to have high AC while wearing no armor or using unarmored defense from either class that does so. Making Baneful your pact weapon should, except for Honor mode, still give 3 attacks per turn. Pop armour of agathys for some reciprocal damage, IF an enemy somehow finds a way to hit you (myrkul sweeps for example). Beginning or before combat start bladesong, then cast haste and mirror image. If you lose your haste someway blur the next turn. Like I said you should have 3 attacks at level 12 before haste, which is slower than fighters, but now you’re level 12 and doing the same number of attacks, AND you’re functionally impossible to hit AND you can still cast all your spells. All told if you bladesong, enter combat, haste and mirror image, once you enter and on your turn (should be first, with your dex) you’ll be sitting at 42 AC, Crit immunity, and enemies have disadv with blur, if you use a haste potion, which there is utility to since its a bonus action rather full action, but it doesnt last as long though it doesn’t take your concentration. If you like drugs, terazul gives another +2 ac and another action because it stacks with haste. Steel watchers are still a pain because of their area attack so deal accordingly, but you should be good for the Dex saves. You’re still gonna take half damage on things requiring saves because casters don’t get evasion. Sorry couldn’t work that in.
