He began his professional career in St. Louis, Missouri in the early 1940s. After playing in a Navy band during World War II he played with Charlie Barnet, Count Basie, Duke Ellington, and Quincy Jones, and then joined the Tonight Show Band. He also performed and recorded regularly both as a leader and sideman, particularly with the quintet he co-led with Bob Brookmeyer.
He is known for his dialogue solos, in which he plays responding phrases on open and muted trumpet or on trumpet and flugelhorn, and for his incoherent scat singing, which originated as a parody of older blues singers. Examples of both can be found on Oscar Peterson Trio + One (1964), an album for which both he and Peterson are renowned.