The Fair-Rutherford House, a Greek Revival cottage, stood here from ca. 1850 until it was demolished in 2004. Built for Dr. Samuel Fair, it passed through several owners before 1905, when William H. Rutherford (1852-1910) bought and enlarged it. Rutherford, an African-American businessman born a slave, taught school, then made lodge regalia and supplies and briefly co-owned a local cigar factory.
The Rutherford House was built in 1924-25 for Carrie Rutherford, daughter-in-law of W.H. Rutherford. Her son Dr. Harry B. Rutherford, Jr. (1911-1980) and his wife Dr. Evaretta Sims Rutherford (1910-1978) were prominent educators, he as a teacher and principal and later a dean at Benedict College, and she as a professor and department chair at Benedict College and Howard University. The house was listed in the National Register of Historic Places in 1984.