After schooling in England, George William Fairfax returned to Belvoir to live in 1746, and married Sarah Cary, also known as Sally, in 1748. They had no children. Upon his father William Fairfax's death in 1757, George William inherited Belvoir. His father's will arranged for the subdivision of his holdings and in addition to the estate grounds, George William received 3 slaves. To Sarah Cary, George William's wife, he left one young slave and her daughter. Records show that Sarah Cary was the child's Godmother.
The remainder of William Fairfax's property and slaves were willed to John Carlyle, his daughter Sarah's husband, and to his children Bryan, William Henry, and Hannah.
In 1759, George William inherited Toulston Manor in Yorkshire, England, birthplace of his father. George William and Sarah spent three years in England from 1760 to 1763 and again returned to England in 1773 to settle inheritance issues. They never returned to Belvoir.
In 1774, they arranged to sell all of their possessions. George Washington purchased many of the fine furnishings from Belvoir, including tables, chairs, rugs, window hangings, and mirrors.
After the Fairfax family moved, the house was rented by a Reverend Andrew Morton who lived there until 1783 when it was destroyed by a fire.
"It is a matter of sore regret, when I cast my eyes toward Belvoir, which I often do, to reflect that the former inhabitants of it with whom we lived in such harmonious friendship, no longer reside there, and that the ruins can only be viewed as the mementos of former pleasures."