Born in New York City, the great-grandson of oil tycoon John D. Rockefeller and nephew of future vice president Nelson A. Rockefeller, Jay Rockefeller graduated from Phillips Exeter Academy in 1954. He graduated from Harvard University in 1961 with a B.A. in Far Eastern Languages and History, after having spent three years studying Japanese at the International Christian University in Tokyo.
After college, Rockefeller worked for the Peace Corps in Washington, DC where he served as the operations director for their largest overseas program in the Philippines. He continued his public service in 1964-65 as a VISTA volunteer, during which he moved to Emmons, West Virginia.
He was elected to the West Virginia House of Delegates in 1966, and to the office of West Virginia Secretary of State in 1968. Following his term as Secretary of State, he served as President of West Virginia Wesleyan College from 1973 to 1976.
He was elected Governor of West Virginia in 1976 and re-elected in 1980. In 1984, he was elected to the United States Senate, and re-elected in 1990, 1996 and 2002. He was chair of the Committee on Veterans' Affairs (1993-1995; January 3-20, 2001; June 6, 2001-January 3, 2003).
Since 1967, Rockefeller has been married to Sharon Percy with whom he has four children: John, Valerie, Charles, and Justin. The Rockefellers reside in Charleston, West Virginia.