Umbar had been built by the Númenóreans during the Second Age, and like the havens of Pelargir and Lond Daer later grew into a fortress from which Númenórean influence spread over Middle-earth. But unlike Pelargir Umbar was mainly used by the King's Men, who were not friendly to the Elves.
Umbar formed a natural harbour of enclosing rock, and it was here that the last king of Númenor, Ar-Pharazôn the Golden, landed and took Sauron captive near the end of the Second Age. In memory of this event a huge pillar was erected at Umbar.
After the downfall of Númenor, Umbar became a major stronghold of the Black Númenóreans, who became known as the Corsairs. Hostile to Gondor and Arnor, they became a menace for southern Gondor in the Third Age, ruling over the nearby Haradrim. Not yet followers of Sauron, the Corsairs slowly fell under his influence, but they remained proud and refused to destroy the pillar that remembered Sauron's capture.
Umbar was conquered by Gondor at the end of the first millennium of the Third Age, and a small taskforce was maintained there, under almost eternal siege of the Haradrim. Umbar became hostile to Gondor again when after the Kinstrife, the usurpers fled to Umbar, and for a time it became a Black Númenórean realm again. But Umbar became less and less powerful, and the blood of its people mingled with that of the Haradrim. Eventually it became an ally of Sauron, and the great pillar was destroyed.
During the War of the Ring a fleet of Corsairs sailed up the river Anduin and was raiding Lebennin, when it was captured by Aragorn, who used their ships to come to the aid of Minas Tirith.
During the Fourth Age, Umbar fell under the control of Gondor, and was ruled by people loyal to King Elessar.