Born in London, the son of actor Sidney Lawford and his (eventual) wife, Mary Aylen, née Somerville, he spent his early childhood in France, and began acting as a child. He became an American citizen in 1960.
He married four times. He married his first wife, Patricia Kennedy, sister to President John F. Kennedy, in 1954. They had four children, including actor Christopher Lawford. They divorced in 1966. He married his scond wife, Mary Rowan, daughter of Dan Rowan, in 1971. They divorced in 1975. He married his third wife, Deborah Gould in 1976. They divorced in 1977. His fourth wife, whom he married in 1984, was Patricia Seaton.
Lawford died in Los Angeles in 1984 of a heart attack and was originally buried in L.A. at Westwood Village Memorial Park Cemetery before his remains were cremated and scattered in the Pacific.