Tolkien's writing do not provide information on more than the first few centuries of this age, so we don't know when it ended, if it ever did.
Tolkien said that he thought the distance between the end of the Third Age and the 20th century was about 6000 years, and that in 1958 it should have been around the end of the Fifth Age if the Fourth and Fifth Ages were about the same length as the Second and Third Ages. He said, however, that he believed the Ages had quickened and that it was about the end of the Sixth Age/beginning of the Seventh. That implies that maybe the fall of Nazi Germany ended the Sixth Age in 1945, as the other ages ended with the downfalls of tyrants. The estimation of 6000 years would place the end of the Third Age and the start of the Fourth in the 5th millennium BC. Since Tolkien was a devout Catholic, it is speculated that he would have held the birth of Jesus Christ to be the most important event of the era, and thus would be the most likely time for the start of the Fifth Age.