Marable was born in Paducah, Kentucky and learned piano from his mother. At the age of 17 began playing on the steam boats plying the Mississippi River. He soon became bandleader for boats on the Strekfus Line which ran several paddlewheelers which held dances and excursions along the river from New Orleans, Louisiana to Minneapolis, Minnesota. Marable appreciated the new "jazz" sound being played by the New Orleans musicians, and the bulk of his bandmembers were recruited from that city.
Members of Marable's bands were expected to be able to play a wide variety of music, from hot numbers to light classics, both play by head and from sheet music, and above all keep the dancers happy. Noted musicians who spent time in Marable's bands included Red Allen, Louis Armstrong, Baby Dodds, Johnny Dodds, Pops Foster, Al Morgan and Zutty Singleton, among others.
In addition to piano and bandleading, Marable played the boats' steam calliope, a contraption that could be heard for miles up and down the river and poured down so much water from condencating steam that Marable performed wearing a raincoat and hood.