Well, that's a lot of questions, Larry, and if I knew that I knew the answers, I'd probably be either dead or rich, depending on how much integrety I had...
Anyway, here's what I think, not necessarily "connected" in the end, but at least the parts of the puzzle as best I can see.
1) The timeless/fundamental human conflict between the quest for unity and the inevitable separation that follows after it is achieved, if it ever is. It's what drives sex, it's what drives non-corrupt religion (and it's what corrupt religion plays off of), it's what drives literature (Seven Basic Themes my ass! There's only one - separation vs unity), it's what drives damn near every aspect of human experience.
2) The uniquely African-American experience of same. Call it "genetic memory", "subliminal cultural heritage" call it whatever you want, but there's no denying that the African "concept" of layered rhythms (some call them poly-rhythms, but that simplifies the technique relative to the end result far too much for my satisfaction) is a key factor in what we call "swing". Due to the circumstances of this experience, what was once a ritualistic/spiritual/whatever deeply meaningful and fundamental experience of elementary oneness morphed into one of "entertainment" (which is not to say that it was in anyway cheapened, just to say that the function of the experience took on new clothes that, no matter how stylish and wonderful they might have been, were at some level different than what they originally were). You gotta wonder how somebody with the unique perceptual genetics of Bird would've sounded if he (and, of course, his ancestors) had lived in an African milieu and been able to express what he saw/heard/felt about life in general without having to do it in a "commercial" environment that was not at all relevant to the root of his perception. Or maybe it was - maybe the American experience gave Bird something unique to react to that he'd not have had otherwise. Who knows? But this is relevant to Trane's times in that Afro-centrism had been "in the air" for quite a while, and was really to the fore in the mid-late 60s. So, you gotta think that this was on his mind in some form or fashion, if only as a factor in shaping the personal nature of his quest (and a quest it certainly was). Not the only factor, of course, but a factor that somebody like Messiaen would not have had a, uh... deeply personal interest in.
3) Albert Ayler - Now, here you go. Here's a guy who took a look around at his musical landscape and said "fuck it, there's more to it than this", and rather than putting in a lifetime of attempted transcendence, just went on ahead and went there. Didn't ask anybody's permission, didn't see the need to "evolve the tradition" or some such, just saw that it was there and didn't see any good reason whatsoever why he shouldn't just go on ahead and take it. And he did. Trane was hip, very hip, to the deepest implications of Ayler. Check out his comment about Ayler dealing in the "upper partials of energy" or some such, a comment which is equal parts Einstein and metaphysics in both it's revealing of what was the process and what was the goal.
The goal was that elusive "oneness", "unity", whatever, the experience of a totality so real that you have to "cease to exist" from a perceptual standpoint in order to experience it (contradictory in a lot of ways, I know, but there it is nevertheless...). The means? A refusal to break down time and space into component parts, to instead be all/everything at once. That's the way things really are, right? Only the limits of our "perception" makes it appear otherwise/ So why shouldn't music reflect that? Shouldn't be all that hard...
Unless you've spent a lifetime or fifty living and perceiving otherwise, which most all folks do. A professional musician is going to live even more deeply in a world where subdivision matters - metronomes, scales, arpeggios, all the tools of "perfection" are based on separating the whole into component parts and then reassembling them in a manner that strikes somebody's fancy. An obsessive musician like Trane is going to be so far deep into this zone, albeit, most likely, in a quest to get past it, if and when whatever "past it" really is would become apparent, that it literally becomes all they know - take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble, take apart, reassemble ,take apart, reassemble, take apart, reassemble, take apart, reassemble, each disassembly becoming more and more micro and each reassembly becoming more and more an attempt at creating the macro.
Well, imagine the shock that Trane must have felt when he heard Ayler and realized that there was really no need to take anything apart (and therefore no need to reassemble it), that all you had to do was get the whole damn thing in its natural form and let it be what it is. Of course, it's not that easy, not by a long shot. you gotta be able to handle the whole damn thing in its natural form, precisely because it is what it is, and people like Ayler & James Brown are proof that you can only hold it for a little while before it fucks you up, whereas people like Warne Marsh are proof that you can hold for as long as you like if you're prepared to submit and be its totally submissive bitch and live by its whims, not yours, not ever, for as long as you can draw a breath. But we get ahead of ourselves...
Anyway, I hear Trane's last year or so as, at least in part, an attempt to get to where Ayler already was in terms of "perception" with the tools he had, which were, to put it mildly, massive (the depth of the structures, micro and macro, in the playing on Interstellar Space is damn near supra-human, as is the instrumental facility which is used to create them). The irony, of course, is that as fine of a saxophonist as Ayler was (and he was a damn fine one in terms of knowing the fundamentals of the instrument as deeply as they could be known), he didn't have anywhere near Trane's overall knowledge of music and saxophone technique. The cross-irony is, of course, that it just didn't matter. Ayler's vison was whole, fully formed, and, for a little while any way, expressed likewise - as is, everything is everything, it's already here, what's the big deal? IT"S OURS BY RIGHT OF BIRTH!!! TAKE IT!!!
Trane's vision was always in the process of forming and/or reformiong, the whole take apart/reassemble thing. So, I think he set about trying to forget all about the taking apart and instead decided to focus entirely on the reassembly. Well, he had taken apart so much that putting it all back together, really putting it all back together, in a form that had no suture marks, was flawless and organic, was nowhere near a simple task. There were bound to be (and were) moments of awkwardness. There were bound to be (and were) moments of stunning brilliance that nevertheless didn't accomplish the ultimate goal. And there was bound to be, and was, at least one moment were the goal was reached, the vision fully fulfilled. That moment was Interstellar Space.
Why the obsession with getting it "right" (or what Trane had come to see as right given all of the above factors?
4) Trane's sense of imminent demise. Much has been written about this, how as early as 1963(?) he had begun to feel that his time here was limited. Draw you own conclusions. And also allow for the possibilty that Trane's acid use was an attempt to "get there" as quickly as possible, the luxury of a long-ish life not necessarily being a possibility that he entertained for himself.
Now, as to whether or not he needed/wanted it to "swing", imo, he felt that it was going to swing because of what it was, which was everything at once. A part subsumed back into the whole doesn't cease to be what it is, it just becomes what it is in a different form. Purple is purple, at least as we percieve it. Do red & blue stop being red & blue in purple? No, they're still there. Purple couldn't exist if they weren't. And when green keeps the blue, where does the red go? To yellow to make orange? Or to lunch with its agent? It's all light, dig, and none of it ever goes away, it just gets filtered and shit so that we think we see colors when what we really see is light. Now, I think that Trane was looking for the light, not the colors, if you know what I mean. If that's the case, then "swing" would have to be there, because that's part of the spectrum. But if you see one color, you don't see the pure light, and the pure light (or energy, or wholeness, or whatever) was by this point what Trane was looking for. Does that make any sense? Does any of this make any sense?
Like I said, the parts of the puzzle as best I can see, not necessarily "connected" in the end. Which is, I suppose, considering the matter at hand, fitting.
At least for a while...