Broom Hall Plantation, later called Bloom Hall and still later Bloomfield, was first granted to Edward Middleton in 1678. By 1710 this property passed to Benjamin Gibbes (d. 1722), who named it for Broom House, his ancestral home in England. Gibbes's widow Amarinthia later married Peter Taylor (d. 1765), a longtime member of the Royal Assembly and rice planter, who built a large two-story brick house here.
Broom Hall passed to Thomas Smith (1720-1790), member of the Royal Assembly and the Provincial Congress. An 1828 visitor called its gardens "the most interesting spot I have seen in Carolina." Sold to Henry Arthur Middleton in 1856, the house was virtually destroyed by the Charleston earthquake of 1886. Later owned by Westvaco, Broom Hall was subdivided for residential development in 1980.