"most people" is an assumption that may be rooted in not paying attention to changing realities.
It was 1986-1987 before RCA began reissuing Rollins in earnest. That was a lot of time for the narrative about the 50s material to take root, even though they weren't all gems.
But the 60s were huge for Rollins creatively, and people are figuring this out. Still.
My Milestone overview thread is still coming. I know that "most people" don't prefer that stuff. I don't either. But I still like it, listen to it often. and hear in it a continuation of the ongoing Rollins arc in terms of tenor playing. In terms of record making, maybe not so much. But in terms of tenor playing, if you don't know (or get) 60s Rollins, you really might be lost by what came later.
Taste is indeed subjective but the realities/specifics of music are not.