The noble Lanthieri family, originally from the Brescia area, moved to the Goriška region in the second half of the 15th century. The mansion in Vipava, which Valvasor calls New Castle, was built around 1659 by Anton V. Lanthieri: this is evidenced by the year carved on the keystone of the portal leading from the palace to the bridge behind it. The cause for its construction was probably the announced arrival of Emperor Leopold I, who on 17 September 1660 actually spent the night in Vipava while on his way from Ljubljana to Gorizia. It was written in the privileges granted to the Lanthieris upon their promotion to the rank of counts in 1642 that every ruler who comes to the country and stops in the main square had the right to reside in the palaces and castles of this family. Documents kept in the family archives show that the Viennese court usually sent special commissioners in advance to arrange for additional stables and apartments to be built in Vipava and Branik for the ruler's sizeable entourage: stables for 240 horses of the military escort were built in Vipava at state expense for this occasion.
Fleshy leaves in the window walls, books bound into bouquets, imaginary landscapes, marbled patterns and awkwardly painted figural scenes with interesting iconographic content taken from ancient mythology and fables reveal the authorship from the same painting workshop that painted the interior walls of the nearby Lanthieri Villa in Zemono in the mid-1670s. In the spirit of baroquisation, the mansion was converted into a country manor house around 1762. In front of the mansion there was a large horticultural park with statues of putti. New outbuildings were built along the park, and the old ones on the island behind the mansion were rebuilt and extended around 1767 to form a symmetrically arranged, three-winged enclosed building complex. The spatial axis that runs from the park through the entrance hall of the mansion and across the small bridge to the courtyard behind it, continues here through the newly built and slightly suppressed rustic portal to another, now demolished small bridge over the source of the Vipava to the place where a pavilion and wine storage once stood. The Lanthieris started gradually abandoning the mansion after the First World War, as it was almost devastated when it housed a field hospital. After the Second World War, the complex was nationalised, and premises in the mansion were given to a cultural centre of the Yugoslav People's Army. After 1991, the complex was taken over by the Municipality of Vipava. The residential building of Lanthieri Mansion was renovated at the end of 2012. Since the beginning of 2013, it has housed the Nova Gorica College of Winemaking and Viticulture.
In addition to the emperors Leopold I and Charles VI (1728), the Lanthieris also hosted other famous persons in Vipava, such as Venetian comedian Carlo Goldoni and painter Rosalba Carrierara. The mansion in Vipava was a major administrative, cultural and social centre of the upper Vipava Valley.