Kanji is actually way faster for speedwriting

Actually, people overcomplicate this. Kanji is ALWAYS faster when taking quick notes because fewer total pen strokes are needed. Your hand moves less distance. Sure, complex kanji have 15 strokes, but kana takes three distinct characters which drag out your wrist movement. Honestly if you're switching back to kana for speed, you just haven't memorized the muscle memory yet. Just write kanji for everything.