This 1906 jail was built to replace the old rock jail that stood behind the courthouse. The rock jail had replaced the first county jail, a two-story log building. Dr. William B. Tate urged the construction of the jail as a grand juror and on two Citizens Committees. The architects were J.W. Coluke and Co.; contractors were William L. Landrum and son. The steel work was installed by the Pauly Jail Co. of St. Louis, Mo., specifically by Luthor Cartwright, who while here married and, eventually settled in Jasper. He later supervised construction of the pink marble mansion in Tate. The work on the front of the jail was done by Lee W. Prather, a local stone worker, using marble from the Delaware Quarry at nearby Marble Hill. The Delaware Quarry was opened c. 1840 by pioneer marble entrepreneur Henry Fitzsimmons.
The jail still has a gallows (non-functioning) although no one was ever executed on this device. The most regular residents of the jail were the sheriffs and their families, who lived here rent free. The jail was ordered closed by a federal court order in 1980. The well in the rear, now filled in, provided water for the hundreds of people who came to Jasper for singing conventions.