Soon after his graduation from university, Holt joined the United States Navy and served on board the USS Barbero, a submarine that fought in the Pacific Ocean. During the war he came to the conclusion that nuclear weapons were the world's greatest danger, and only a world government could prevent nuclear war. After his three-year tour of duty, he got a job with the New York branch of the United World Federalists. Starting in the mail room, he became the executive director of the New York branch within six years. However, he became frustrated with the group's ineffectiveness, and he left the organization in 1952.
At the urging of his sister, Holt became a teacher. After several years of teaching in Colorado, he moved to Boston. It was here that he met Bill Hull, a fellow teacher, and they decided to start a classroom observation project; one would teach, while the other would watch. The notes and journals entries Holt accumulated during his first eleven years of teaching formed the core of his first two books: How Children Fail and How Children Learn. These books detailed the foundational ideas of Holt's philosophy of education. He held that the primary reason children did not learn in schools was fear: fear of getting the wrong answers, fear of being mocked by the teacher and classmates, fear of not being good enough. This was worsened, he maintained, by children being forced to study things that they were not necessarily interested in.
After many years of working within the school system, Holt became disillusioned with it. He became convinced that reform of the school system was not possible because it was fundamentally flawed. Thus, he became an advocate of homeschooling. It was not helpful, however, to simply remove children from the school environment if parents simply re-created it at home. Holt believed that children did not need to be coerced into learning; they would do so naturally if given the freedom to follow their own interests and a rich assortment of resources. This line of thought became known as unschooling.
In 1977, Holt founded the nations first homeschooling magazine, Growing Without Schooling and started John Holt's Bookstore.