CD Cellar is nice. But the best place by far for used jazz CDs in the DC area is CDepot in College Park. (9039 Baltimore Ave. College Park, MD 20740) If you have the time, I recommend going up there. It's as big as it looks, and the jazz section stretches for the full length of the store. Box sets are also abundant.
If you are only going to be around Dupont Circle, you will probably want to check out Melody Records on Connecticut Avenue. They are one of the few genuinely indepenent record stores (for new records) in DC, and they do fairly well at ordering good jazz. For example, they stock Nessa!