He married Married Lady Jane Fitzalan-Howard (daughter of the 16th Duke of Norfolk) in 1975. Previously an Advocate on the Scottish Bar, he unsuccessfully contested the East Lothian parliamentary seat in 1970. He won Berwickshire & East Lothian in the February 1974 election, only to lose it again in October, then represented Edinburgh South during the Tory high years of 1979-87. He has been Member of Parliament for Devizes in Wiltshire since April 1992.
He was a member of the House of Commons Energy Select Committee between 1979 and 1983 and Chairman of the Scottish Conservative Party from 1980 to 1983. He was Parliamentary Under-Secretary of State at the Scottish Office with responsibility for Home Affairs, Housing, Local Government, Rating Reform and the Environment from 1983 until 1987.
He was a member of the Public Accounts Committee and Chairman of the Backbench Constitutional Affairs Committee from 1992 until May 1993, when he was appointed Parliamentary Under-Secretary of State at the Northern Ireland Office. He was promoted to Minister of State at the Northern Ireland Office in January 1994, and was made a Privy Councillor in January 1996.
After the Conservatives' defeat at the 1997 election he served in the Shadow Cabinet as Constitutional Affairs Spokesman (June 1997 - June 1998) and as Chairman of the Conservative Party (October 1998 - September 2001).
In 2001 he stood against Iain Duncan Smith, Michael Portillo, Kenneth Clarke and David Davis for the party leadership. In the first poll he and David Davis were tied for last place, leading to a re-run in which Ancram was placed bottom and he was eliminated and Davis withdrew. Both swung behind Iain Duncan Smith who went on to win, beating Clarke and Portillo in the second round. Duncan Smith made Ancram Deputy Leader of the Opposition and Shadow Secretary of State for Foreign & Commonwealth Affairs in September 2001. He remained in this position after Michael Howard took over in 2003.