Agree completely. Puerling seemed to be more about reharmonizing with the Singers Unlimited, and creating a lot of sustained, impressionistic sorts of chords and effects. Yes, I hear the "youthful arrogance" in the Hi-Los. My dad had some of the really early stuff from the Trend label (45s like "They Didn't Believe Me") but also some of the early Columbia LPs, mostly with (the great) Frank Comstock. My Dad (who was a WWII era guy and a professional group singer) also said that the Columbia stuff wasn't as good as the earlier stuff. I always liked the Columbia stuff just as well, and "The Hi Los and All That Jazz" has always been a favorite.
Luckily, I've inherited my Dad's records, though I wish he were around to appreciate them with me.
I should also add that I listen to SI and the HLs when I'm in completely different moods. SI would be good hangover music, but listening to the HLs with a hangover would be torture.