Davis studied the clarinet at the Royal College of Music in London, where he was barred from taking conducting lessons owing to his lack of ability at the piano. Nontheless, he formed the Kalmar Orchestra with fellow students, and often conducted it.
In 1952, Davis worked at the Royal Festival Hall, and in the late 1950s conducted the BBC Scottish Orchestra. He first found wide acclaim when he stood in for an ill Otto Klemperer in a performance of Wolfgang Amadeus Mozart's opera, Don Giovanni, at the Royal Festival Hall in 1959. A year later, he stood in for Thomas Beecham in similar circumstances in Mozart's The Magic Flute at Glyndebourne.
In the 1960s he worked at Sadler's Wells Opera, the London Symphony Orchestra and the BBC Symphony Orchestra. In 1971 he succeeded Georg Solti as principal conductor at the Royal Opera House, where he had given occasional performances before, remaining there until 1986. He became noted for championing the operas of Michael Tippett, giving the premieres of his works The Knot Garden (1970), The Ice Break (1977) and The Mask of Time (1984). In 1977 he became the first English conductor to appear at the Bayreuth Festival (dedicated to the works of Richard Wagner) where he conducted Tannhauser.
Davis was knighted in 1980. He subsequently worked at the Bavarian Radio Symphony Orchestra and the Boston Symphony Orchestra before being appointed principal conductor of the London Symphony Orchestra in 1995.
Aside from his championing of Tippett and interpretations of Mozart, Davis is particularly closely associated with the music of Hector Berlioz, giving many performances of both his operas and orchestra works.