Would that be Full Moon in Candler Park? I travel to Atlanta frequently and hit Full Moon, Wuxtry, Criminal and Wax 'n' Facts while in town.
St. Louis has two biggies: Euclid Records and Vintage Vinyl. Euclid has an ungodly selection of Jazz... the primo stuff at ungodly prices. Vintage usually has a few gems tucked away at reasonable prices. Record Exchange is also huge but the selection feels like all the records that Euclid and Vintage didn't want. There are a ton of mostly CD shops as well.