Although I think it's overstated by a good amount, I've actually agreed with the "Jerry Lewis is a genius" thing at root. The guy gets timing like very few others, and not just in terms of payoffs. His extended setups unfold impeccably on their own, and the sheer appearance of his props are always where they need to be to get the groove started, like how Morris Day and The Time hooked you in just by looking at them, before anybody even moved or made a sound. The gag started at first subconscious sight, even before your mind had time to process the information. Jerry Lewis is/was like that as well.
Is his humor "imbecilic"? Sure, and it's quite often not particularly "funny". But that's only if you think about it, and some things are better if you don't think about them, at least not right away.