Mingus did walk out over Max's drumming. You will recall that he only liked Danny Richmond at the time, and would only use him on his recordings.
But Duke must have disliked Mingus, because when the personnel for Mingus's concert in 1971 was being chosen (issued by Columbia, and it's great too), the Duke heard that Mingus wanted Harry Carney, and actually paid him not to appear. So, Charlie had to settle for Gerry Mulligan, lol.
How sad, when you consider how much Mingus loved Duke's work.