Oh, it could matter a lot in simple (or not so simple) human terms if Jackie sounded the way he did and was white, or if he sounded the way he did and got a lot of guff from other black guys about the way he looked, but we know the first isn't so, and I've never heard anything about him getting hassled on the second count.
It mattered to me in high school (small farming community in central Iowa). The Prestige records I had at the time used photos that were "one or the other" and he sounded unlike white players I had heard at the time. At that point in my life, when I was trying to figure out 1000 things, this was one.
Interesting (in a good way) to have this thread resurrected now.