You may want to expand your sources. That has been disproven. The University of Arizona has all of Baxter's existing arrangements. Many of these are in Baxter's own hand; others are in the hands of his assistants, Hall Daniels or Albert Harris.
Albert Harris, who was a highly respected arranger who worked with just about everyone in Hollywood, said that Baxter's sketches were very detailed, and regardless of which arranger did the grunt work, the end results would have been the same.
Albert Harris composed and arranged an orchestral suite for Frank DeVol, titled Bacchanal. Listen to it sometime. It is gorgeous, but it sounds nothing like Baxter's exotica albums.
There is a consistent thread in all of Baxter's exotica works, from Voice of the Xtabay in 1950 to Que Mango in 1970, and everything in between. Same with his gothic scores for AIP films. That can't be a coincidence.