Swan was born in Cullman County, Alabama. He ran away from home when he was thirteen or fourteen and ended up in Wayne County, Mississippi. Swan sang in nightclubs and wrote songs until he became one of the earliest disc jockeys in the country. He joined with several others in founding the Country Music Disc Jockeys Association. Swan performed for many years with his own radio program, and eventually became part owner and manager of a radio station. In the 1960's, he became deeply concerned over the developments in the United States and especially the state of Mississippi. In 1967 and 1971, Swan unsuccessfully ran for governor of Mississippi.