The cluster of fortified buildings that developed on the hill in medieval times, was first identified as the castle in 963 AD, in a document by which Emperor Ottone I donated the lands around the castle of Polcenigo to Giovanni, Archibishop of Belluno. Between the XI and XII century the castle and the fiefdom were handed down to the Signori di Polcenigo, later appointed counts, who inhabited the place till the first half of the XIX century.
At the beginning of the 18th century the structure was crumbling to pieces. In 1738 the counts commissioned the Venetian architect Matteo Lucchesi to build a comfortable luxurious villa in Venetian style that had to replace the austere fortress and become their new residence.
Salvage materials from the ruins of the old castle, together with stones and boulders from the nearby quarries, were used in the construction of the elegant mansion. The result was an imposing building structured on three levels that reminded the palaces overlooking the Grand Canal in Venice, with plenty of rooms, a dance hall, fireplaces, stuccos, tapestries and mirrors.
The main entrance led to a great hall where guests were entertained and balls took place. Each area of the building could be easily reached from the hall: the staircase leading to the bedrooms, the kitchens, the prisons and also the inner yard leading to the back garden. The dance hall was connected to a flight of 365 majestic steps (more than 10 metres wide) leading directly into the village. The main garden admitted to the nearby St Peter’s Church, the counts’ private Church.
The villa was inhabited for about one hundred years then, following hereditary feuds among the counts, the structure was abandoned and slowly deteriorated. The roof and the floors caved in and the stone and marble works were either removed or sold to be used in the construction of other buildings (the stone steps, for example, were used in the parish church in Vigonovo).