Well, ok, but even at that, listen to the tone he gets out of his kit on those louder dates - it's still "soft" in comparison to a Blakey, or an Elvin, or a Max.
Billy's kit did not seem to be tuned in such a way as to even be capable of being "loud", at least not in a relative sense. Same with his cymbals - his overtones were always controlled, and never turned into white noise.
How he's recorded and mixed (Rudy mixed on the fly, remember) has as much to do with it as how he was playing. He's recorded really hot on GO!, for example. And on THE GIGOLO, yeah, he's playing harder than hell, but the actual sound of the kit isn't really "loud" - it's the recording that captures (brilliantly, imo) the power of his playing. But if you put his playing there in the same room with Blakey on a normal day, Blakey would probably drown him out, figuratively if not literally.
Records don't sound like live playing, even the best ones. You gotta learn to "translate" what you hear on a record to grasp what it most likely really sounded like in the studio. Throw in all the analog-to-digital transfer funkiness, and all bets are off.