One of the follies erected by 'Mad Jack' Fuller around Brightling, the sugar loaf was supposedly built to win a bet.

Whilst in London Fuller argued with a friend that he could see the spire of the church in a nearby village from his home. When he returned home he discovered that the spire was not visible, and quickly had the Sugar Loaf - whose shape closely resembles that of the spire - built on a ridge of hills between his home and the village.

The Sugar Loaf gets its name from the way in which sugar was supplied at the time - in cones called loaves.