Was Big Al sears living in Baltimore at this time? Listening to this version, it sounds like a pick up group of top whack pro jazz musicians, not a bunch of local guys. The rhythm section in particular is so loose and groovy it just swings the hell out of the Bryant version, which is fairly strict tempo.
Maybe Brown DID form a band in Baltimore but it sounds like when they got to the New York studios, there were all these other guys who'd been assembled by a producer who knew what was what.
MG
I'm pretty sure Al Brown was New York based, and the players on his version are all NYC session cats. Brown's name can be found as leading the backing band on a number of R&B sides from the late 50's and early 60's. Brown's record was definitely the hit Madison tune in the Northeast at the time. I never heard Bryant's record until years later when I had gotten into jazz. I was in high school when the "craze" happened, and nobody ever danced the Madison at any dances I attended! I believe it was a Baltimore fad, and has already been suggested.