Eldridge was born in Pittsburgh, Pennsylvania. His nickname was Little Jazz. Stylistically, he was a link between Louis Armstrong and Dizzy Gillespie. Eldridge played in the bands of Fletcher Henderson, Gene Krupa and Artie Shaw before making records under his own name. He also played in Benny Goodman's and Count Basie's Orchestras, and co-led a band with Coleman Hawkins.