Funny you should use the word "difficult". I remember, back in my college days when I was just discovering Patton, thinking at first that he was a simplistic organist with a good groove. Then one night, it hit me - the individual components of what he was doing were relatively simple, but how they all fit together, not just on his instrument, but with the entire group, especially with the drummer, were actually pretty darn involved. Not obviously "intricate" necessarily, but in no way simplistic. There was a lot of thought going on, and more importantly, an overriding vision to how the the totality of the music was going to sound. The degree of specificity involved was (and remains) pretty amazing.
That's the kind of music I really dig, regardless of "genre" - they type where everything that everybody plays is exactly what it needs to be to create a specific final result. No wasted motions or grandiose "gesturing". Just make a point that's as direct as it can possibly be. That's not a function of "style" or "technique" either, because I've heard highly specific music that is alos very "busy". What it is a function of is clarity of vision, and John Patton certainly had that in abundance.
To that end, the Nilva side, where he hooked up with another master of specificity, Grachan Moncur III, was a match made in heaven!