The town was almost destroyed by a great fire in 1688.
The Butter Cross was constructed in 1689. It was the place where farmers displayed their butter, eggs and other farm produce for sale. Until 1810, there was also a Corn Cross, but it was taken down and replaced by a pump.
What was once the 12th century church of the Benedictine Priory (founded by Gundreda, wife of Roger Bigod), is now the parish church of St. Mary. A wooden panel behind the altar was presented to the church by the novelist Sir H. Rider Haggard who was born nearby in Bradenham near Kessingland and lived in Ditchingham.