This house, an excellent example of early Federal era-architecture, was built about 1800 for Joseph Koger, Jr. (1779-1866), planter, state representative 1806-1812, Colleton District sheriff 1813-18, and state senator 1818-1838. Koger moved to Mississippi in 1838 and sold the house to his brother-in-law, John Soule Murray (1792-1844), planter and state senator 1840-43.
In 1865 James Parsons Carroll (1809-1883), chancellor of the state court of equity, bought the house as a summer retreat. Carroll had been a state representative 1838-39, state senator 1852-53 and 1858-59, and a delegate to the Secession Convention. The house has long been called "the Old Carroll Place" and was listed in the National Register of Historic Places in 1974.