By the C13th a settlement called Castelmoron had become established, grouped around a castle, located by a meander of the river. The village did not become transformed into a Bastide, as happened elsewhere.

Early in the Hundred Years war the town became English, under the control of the Chevalier de Caumont, whose descendants continued in a prominent role for the next three hundred years. The towns fortifications were demolished when it reverted to France in 1435.

A Protestant centre, the town was affected when the accommodation between the religions was revoked in 1685 and a number of citizens were obliged to leave. The Protestant church was demolished in 1693, but a hundred years later Castelmoron remained unscathed by the Revolution.

Commerce on the river grew to a peak during the C18th, with Castelmoron a focus for the export of wine and flour to Bordeaux and hence Northern Europe, Louisiana and the West Indies. Numerous improvements were made to assist navigation and trade on the river during the C19th, including a a suspension bridge in 1845 and a reconstructed quayside in 1875. However, the establishment of a railway line spelt the death-knell for waterborne commerce and the river was withdrawn from navigation in 1923. 

The 1845 bridge position and its access roads meant that the existing C13th church was rebuilt as part of a radical change to the configuration of the once medieval town, perpendicular to the river rather than alongside. The ancient castle was transformed by Madame Solar in 1871, according to the wishes of her late husband, inspired by visions of Moorish Spain – the Alhambra and Cordoba. The resultant ‘Solar Castle’ became the town hall in 1902.

The 1845 bridge was rebuilt, in concrete, to designs by the Danish engineers Christiani and Neilsen in 1933.