During the Middle Ages, an estate f the Livonian Order was located at Paju. Later, the manor was associated with the noble families of von Lode, von Ditmar, von Buxhoevden and the Freytag von Loringhoven. The single-storey stone main building was probably completed at the beginning of the 19th century. During the Soviet period it was reconstructed for use as a collective farm and it has remained like than until now (privately owned).