Leinsdorf was born in Vienna, and studied music there. From 1934 to 1937 he worked as an assistant to Bruno Walter and Arturo Toscanini at the Salzburg Festival. He conducted at the Metropolitan Opera in New York City from 1938, being particularly noted for his Wagner. He was conductor of the Rochester Philharmonic Orchestra from 1947 to 1955, and music director of the Boston Symphony Orchestra from 1962 to 1969. He subsequently appeared as guest conductor with a number of orchestras.
His memoirs, Cadenza: A Musical Career, were published in 1976.