None to add, but a few 'seconds':
Jazz Record Center - best for vinyl, but some fine CDs, especially in the sale racks.
Downtown Music Gallery - I wish everything was filed/shelved, but a great, great store, especially for AG/'downtown' scene.
J&R for the mor common stuff.
Don't think it's been mentioned, Other Music on E4th. Not a huge jazz collection, but some decent stuff, along with other ear-opening items.
Finally, just cruise St. Marks.