The Elms, an inland rice plantation on the headwaters of Goose Creek, was owned by the Izard family for more than 150 years. In 1704 Ralph Izard (d. 1711), member of the Commons House of Assembly, bought a 250-acre tract here, expanding it to more than 500 acres. His son Ralph II (d. 1743) also served in the Assembly and on the Royal Council. The first to plant rice at The Elms, he enlarged it to more than 2,700 acres.
Ralph Izard III (1742-1804) lent money to the Patriot cause and later served in the Continental Congress. A state representative after the war, then U.S. Senator, Izard was briefly President Pro Tempore of the Senate. An 1818 visitor to The Elms described its "avenue of lofty elms & of loftier live oaks." Its ca. 1718 house, later rebuilt after a fire, was virtually destroyed by the Charleston earthquake of 1886.