I really understand where you are coming from. I can't exercise (or maybe that should be 'I choose not to exercise') because when I get out of breath in the slightest I have a panic attack. It all started a few years back when I was under a huge amount of stress and I got out of breath helping a neighbour try to push start her car. I had a panic attack because my throat was tight (stress) and I thought I couldn't get enough air and had to go indoors and lay shaking under the covers. I had a couple of things in my past which may be the triggers - my dad throttling me round my throat when I was 'naughty' once and then later almost drowning in the sea - both very traumatic experiences for a 5-6 year old.
Anyway, after this first panic attack my stresses continued and so did the panic when getting out of breath. I think it has become a learned response for me. I get out of breath; my mind says to me 'whenever you get out of breath you have a panic attack, well guess what, you are out of breath now so PANIC' and unfortunately that happens. I can be fine doing a bit of exercise but as soon as I realise I have been doing it for a bit I am like you and I start to worry. I worry that I have overdone it and when I stop I won't be able to get enough air in my lungs and so begins the panic cycle. I am going to take Walt's advice and try laughing at my inner bully.
I hope I haven't highjacked your post. Not my intention. I just wanted to let you know that although it isn't a pain issue for me I understand your fears but it also seems that you are doing really well and should be really proud of your progress.