He was born in Hartford, Connecticut, and was a younger brother of author Dominick Dunne. He suffered from a severe stutter and took up writing to express himself: eventually he learned to speak normally by observing others. He graduated from Princeton University in 1954 and worked as a journalist for Time magazine. He married novelist Joan Didion on 30 January 1964, and they became collaborators on a series of screenplays, including A Star Is Born (1976) and True Confessions (1981).
As a literary critic, he frequently was a contributor to The New York Review of Books.
He was uncle to actors Griffin Dunne and Dominique Dunne.