Lying on the west side of the town of Dorking, the Bury Hill estate was created by Edward Walter. He purchased Chadhurst farm in 1735 and bought up adjoining agricultural lands to build up an estate of 1600 acres which ran from Westcott and Milton to Coldharbour and the fringes of the Holmwood. On his death Bury Hill came into the hands of his son-in-law, Viscount Grimston.
Bury Hill was sold in 1815 in lots. Nearly a thousand acres were bought by Robert Barclay, a wealthy Southwark brewer. It remained in the Barclay family for 150 years. Barclay was a great gardener and botanist and he created pleasure gardens with large ornamental lakes. His gardener went on to found the Birmingham Botanical Gardens.
The house was occupied by the military in the Second World War and the centre part was destroyed by fire on being converted in 1949, though the outer parts remain as apartments. The rest of the estate was then split and sold off.