Albertson was born in Reykjavík, but his father left the family before he was a year old. Yvonne, his mother, married three more times.[1] He was educated in Iceland, Denmark and England before studying commercial art in Copenhagen.
In 1947, while living in Copenhagen, Albertson listened by chance to a Bessie Smith recording on radio; it led to an abiding interest in jazz and blues music. "We found magic in such names as Kid Ory, King Oliver, Johnny Dodds, Bessie Smith and Ma Rainey", he wrote on his Stomp Off blog in 2010.[1] On his home tape machine, Albertson recorded visiting British New Orleans revivalists Ken Colyer, Chris Barber and Lonnie Donegan in 1953. These recordings were subsequently released on the Danish Storyville Records and British Tempo Records labels.