The Robert Steward House was built between 1740 and 1770 by Robert Stewart (d.1776), planter and militia captain; it was acquired in 1787 by Daniel Tucker (d. 1797), prominent Georgetown merchant. When President George Washington arrived in Georgetown during his southern tour on April 30, 1791, a militia company and local reception committee met him at the nearby boat landing and escorted him here, where he spent the night as Tucker's guest.
Washington was entertained lavishly and then honored by the Masons of Prince George Lodge No. 16 and the Winyah Indigo Society.
This house, the only extant brick residence in Georgetown built before the American Revolution, was later owned by Benjamin Allston, Sr. (1765-1847). It remained in the hands of his descendants, the Pyatt and Parker families, until 1979.