His son Sir Hugh (d. 1265) was an important ally of Simon de Montfort during the reign of Henry III. He served briefly as Justiciar of England and as Constable of the Tower of London. He was summoned to Parliament by Simon de Montfort, and so might be deemed a baron, though the legality of that assembly is doubtful. He was killed fighting on de Montfort's side at the Battle of Evesham.
His son Hugh (1262-1326), sometimes referred to as "the elder Despenser", was for a time the chief adviser to King Edward II of England. He was created a baron by writ of summons to Parliament in 1295. He was one of the few barons to remain loyal to Edward during the controversy regarding Piers Gaveston. Despenser became Edward's loyal servant and chief administrator after Gaveston was executed in 1312, but the jealousy of other barons led to his being exiled along with his son in 1321. Edward found it difficult to manage without them, and recalled them to England a year later, an action which enraged the queen, Isabella, the more so when Despenser was created Earl of Winchester. When Isabella and her lover, Roger Mortimer, led a rebellion against the king, both Despensers were captured and executed. The elder Despenser was hung at Bristol on October 27, 1326.
His son Hugh (d. 1326), sometimes referred to as "the younger Despenser", was king Edward's favourite after the death of Piers Gaveston. The king had him married to his niece Eleanor de Clare, daughter of his sister Joan of Acre, and in her right he inherited Glamorgan. His greedy push for greater domination in south Wales was one of the things that roused the barons to exile him and his father in 1321. The pair returned the next year, the younger Despenser was captured along with his father during Queen Isabella's rebellion, and was hanged November 26, 1326 at Hereford.
His son Hugh (~1308 - 1348/9) fought at the battles of Sluys and Crecy. He was created a baron by writ of summons to Parliament in 1338 (the titles of his father and grandfather having been forfeited by virtue of the convictions of treason).