He was born in Philadelphia, Pennsylvania to a family of Irish descent, and ordained a priest on December 15, 1945. He was initially assigned to St James High School in Chester, Pennsylvania. He obtained a master's degree in advanced ethics from Villanova University and a doctorate in political science at the Catholic University of America.
He joined the Navy in 1952 as a Korean War chaplain, rising through the ranks to become rear admiral and chief of Navy chaplains. He was appointed Auxiliary Bishop of United States of America, Military by Pope John Paul II on April 24, 1979 and ordained a bishop on May 27, 1979 with the titular see of Curzola. He was appointed Bishop of Scranton, Pennsylvania on May 6, 1983 and installed in that position June 29, 1983. He was appointed Archbishop of New York, New York on January 26, 1984 and installed in that position on March 19, 1984. He was elevated to Cardinal on May 25, 1985.
As Archbishop of New York, later Cardinal Archbishop of New York, O'Connor was a complex figure. He proved very media-friendly with his effective use of soundbites and his ability to charm journalists. Nevertheless he was a stern critic of New York's political leaders. He was also a close associate of President Ronald Reagan and was an outspoken critic of abortion and gay rights.
In 1999 O'Connor was diagnosed as having a brain tumour. On March 7, 2000 O'Connor was awarded the Congressional Gold Medal. He died of a brain tumour in the Archbishop's residence, and is interred in the crypt under the altar of St. Patrick's Cathedral.