He was the grandson of Pippin the Elder from the marriage of Ansegisel and Begga, the daughter of the Elder. As the Mayor of the Palace of Austrasia, Neustria and Burgundy from 680 to 714, he gradually controlled the Frankish court. The Merovingian king Theuderic III attempted to oust Pepin from his post, but he was defeated at Tertry in 687. Pepin then became the actual ruler of Austrasia, keeping a strong influence over the other Frankish kingdoms. His descendants continued to serve as Mayors of the Palace, eventually becoming the legal rulers of the Frankish kingdoms.
Around 670, Pippin II married Plectrude for her inheritance of substantial estates in the Moselle region. They produced at least two children and through them at least two significant grandchildren. These legitimate children and grandchildren claimed themselves to be Pepin's true successors and with the help of his widow Plectrude tried to maintain the position of Mayor of the Palace after Pepin IIís death on December 16, 714. However, Charles Martel, Pippin's son by his mistress, Alpaida (or Chalpaida), had gained favour among the Austrasians, primarily for his military prowess and ability to keep them well supplied with booty from his conquests. Despite the efforts of Plectrude to silence her rival's child by imprisoning him, he became the sole Mayor of the Palace and de facto ruler of Francia.