No advice for you (sorry), but I can identify with the anxiety. My job is also one I stumbled into, and for the most part it's been okay (I work in state government), but I just found out that I'm being transferred out of my current location, which is just about as good as it can be for me in this organization, into a position that is decidedly less attractive. In fact I could hardly think of a worse one. But when I think about making a career change, like you, I see a lot of seemingly insurmountable obstacles, like years of schooling. Doing music full time is a dream of mine (that I still haven't given up on), but that's not a feasible move right now due to my financial and personal obligations.
But like I said, reading your post I can certainly emphathize with the way you've been going round and round in your mind. Fear, doubt, and worry are constant companions. Again, sorry that I don't have any good advice, as I'm still trying to work it out for myself!