Not as far as I know. You might just be asking the wrong person. Anyway, it's only while you're down there. Here, I'll show you-

[He swings a leg over the edge, calling his magic out in one hand and keeping the other on the ladder to gently float himself down. Once he gets maybe halfway to the bottom, the magic fizzles and vanishes, and he just barely catches himself with a foot on the next rung down. Pulling himself back up for a few moments by hand, he's finally able to call his magic back to rise to the top.]

-See? Comes right back. It's really weird.