In 1654 the Domaine de la Pierre belongs to the Rouzand family. Its ownership is passed to the Fauvel family in 1674 and then to the Allard family in 1762.

The Domaine will then be the property of the Challaye family until 1783, date at which the castle and the surrounding land are passed on to Joseph Florimond de Bronac de Vazelhes, through his wedding to Marie-Anne Challaye.   The castle has remained within that family until today.

With the family focus shifting towards estates in Grézieux le Fromental and Montbrison in the course of the 19th century, the Domaine was left less and less used and was left uninhabited, eventually abandonned and grew slowly forgotten.

Being unoccupied for close to 120 years, the estate eventually turned into a ruin. La Pierre got some much needed attention in the 1980s when Jean de Bronac de Vazelhes, saddened by the fate of this property, engaged into a first phase of renovation works, fixing the roofs and the main walls, hereby saving the buildings. A second phase of renovation works has just been engaged into in 2013, with the aim to completing his undertaking and finally bringing back the castle to life.