He was educated at the classical school of Frederiksborg and the university of Copenhagen. In 1828 he became reader, and in 1829 professor of Latin language and literature at Copenhagen, and in 1832 was appointed university librarian. In 1848 Madvig entered parliament as a member of what was called the "Eider-Danish" party, because they desired the Eider to be the boundary of the country. When this party came into power Madvig became minister of education. In 1852 be became director of public instruction.
Some years later, from 1856 to 1863, Madvig was president of the Danish parliament and leader of the National Liberal party. With these brief interruptions the greater part of his life was devoted to the study and teaching of Latin and the improvement of the classical schools, of which he was chief inspector. As a critic he was distinguished for learning and acumen. He devoted much attention to Cicero, and revolutionized the study of his philosophical writings by an edition of De Finibus (1839; 3rd ed., 1876). Perhaps his most widely known works are those on Latin grammar and Greek syntax, especially his Latin grammar for schools (Eng. trans. by G Woods).
In 1874 his sight began to fail, and he was forced to give up much of his work. He still, however, continued to lecture, and in 1879 he was chosen rector for the sixth time. In 1880 he resigned his professorship, but went on with his work on the Roman constitution, which was completed and published before his death. In this book Madvig takes a strongly conservative standpoint and attacks Mommsen's views on Caesar's programme of reforms. It is a clear exposition, though rather too dogmatic and without sufficient regard for the views of other scholars. His last work was his autobiography, Livserindringer (published 1887).
One of his textbooks, on Latin syntax, has been re-issued as recently as in 2001.