He represented Massachusetts in the US Congress under the Articles of Confederation from 1784 to 1786. King opposed the expansion of slavery into the Northwest Territory and proposed that slavery be prohibited in that area. This prohibition became law in the Northwest Ordinance of 1787. He moved to New York City in 1786 and served as a US Senator from New York from 1789 to 1796 and from 1813 to 1825. He also served as the United States ambassador to Great Britain from 1796 to 1803 and in 1825 to 1826. He was the Federalist Party candidate for Vice President of the United States in 1804 and 1808 and for President of the United States in 1816.
His half-brother Cyrus was also a Congressman, as were his sons John Alsop King and James Gore King.