George H. Yarbrough (1826-99) came to Tyler from Alabama in 1854, and entered into partnership with Col. Alfred Ferguson in a general mercantile enterprise. About 1859, Yarbrough built this hand-made brick structure, the first 3-story edifice in Tyler.
In 1862, Yarbrough, W.S.N. Biscoe, and J.C. Short started the Tyler Ordnance Works, which was sold the next year to the Confederate States of America. This building was leased by the Confederacy for use as a cartridge factory.
In the early 1950s, the upper story was removed, and in 1965, the present facade added.
The Yarbrough Building, which once stood at this location, was razed in 1978.