"most people" have been conditioned to have very limited spectrums of their senses. whether or not this conditioning builds on natural instincts or is in fact a brainwashing of sorts to keep people "in line"....I don't know for certain. Maybe both?
Any way...my "project" the last few years is to figure out what it is about stuff that most people like that I can't stand. And almost inevitable, it comes down to, do I feel an expression, or are people just playing notes to make a familiar sound. I've learned a lot doing this, including that even people who are just making "product" with no real sincerity other than to get that product out can and sometimes do have a real skill set for doing exactly that. Call it "quality bullshit". Listening to contemporary country is prime fodder for this. I can spot the various "triggers" in the music and scourn them while at the same time digging the way the record sounds, all the production decisions and such. and then...shut that crap off.
As for jazz....like i said most people have a pretty small range of comprehension about things like harmony and rhythm. Even tone, I man if you can't tell, say, Grover Washington on soprano from Kenny G....how can that be except you just either don't have or haven't developed some pretty basic discernment skill.
And how I listen to jazz now combines a lot of things, but I start with rhythm and tone/sound. That's where I start from. After that, I'm happy/"happy" to be able to get into the technicalities once that test is passed. But sometimes, fuck it, just let me enjoy it first.