Consider what angered the woman in the first place. She got angry all out of proportion because you made a turn she didn't know about. I would say that the woman got angry out of proportion due to her own feeling of some kind of incompetence navigating the city. Then she took it out on you for "making her angry" by lecturing to her son that it was you who was incompetent with your manners due to the way you were brought up.
Maybe you too got a little angry out of proportion, feeling put on the defensive by her, which was only natural. If your anger was disproportionate to her "threat," then maybe you just could remind yourself more often that you are not racist after all. Once you realized that the problem was with the lady, not you (as Daniel A. pointed out), you could probably easily let it go.
I would say she is definitely confusing her child!
GA Russell's point that this might be generational was also interesting.