In 1844, Richard D. Lee inherited 481.5 acres from his father's estate. For the next 16 years, Lee improved and expanded his land holdings until he owned 2,100 acres. In 1860, his farm yielded 2,900 bushels of wheat, 3,500 bushels of corn, 1,400 bushels of oats, 1,200 bushels of hay and truck crops. In addition, he owned 14 milk cows, 72 beef cattle, 35 sheep and 130 hogs. Lee was a successful scientific farmer who followed the agricultural practices of Edmund Ruffin and John Taylor.
By 1800, Tidewater soil was exhausted from generations of tobacco cultivation. John Taylor's Arator (1813) discusses the rejuvenation of the soil through crop rotation, deep plowing, draining and enclosing livestock. In 1832, Edmund Ruffin's "Essay on Calcareous Manures" details the use of marl for reversing soil acidity and harvesting higher yield crops. Richard Lee and Peninsula farmers practiced these new methods of cultivation, and they formed the Middle Plantation Agricultural Society for the promotion of scientific farming in 1860.
The Civil War brought destruction and ruin for Lee's farm and property. In 1862, the Confederates destroyed 85 acres of wheat and tore down eight miles of his fences. Lee abandoned his land and did not return until September 1865. After the war, he placed 13 liens on his property and owed $30,000 by 1870. Local newspapers advertised the sale of a "Valuable Plantation, in Warwick County, late the property of Richard D. Lee ? This is the largest and most valuable estate in the County of Warwick, and is highly improved." In January 1871, William Henry Aspinwall of New York purchased the property.