In the summer of 1862, Confederate authorities imprisoned four Union men from Fredericksburg and Spotsylvania County. The arrested Unionists were local citizens in good standing, but who refused to renounce their allegiance to the United States. They were imprisoned in Richmond for disloyalty to the Confederacy.
In July and August, Federal authorities retaliated by rounding up nineteen local men, holding them briefly at the Farmers Bank (now the National Bank of Fredericksburg, two blocks to your right), and then sending them to the Old Capitol Prison in Washington, D.C. The Federal response was heavy handed, but the arrest of prominent citizens, including Mayor Montgomery Slaughter and Reverend William Broaddus, had the desired effect and an exchange of hostages came about within two months.
The Old Capitol Prison was initially a Washington D.C. boarding house. Located at the corner of First and A Streets, NE, it had served as the nation's capitol building (hence the name) after British troops sacked the city during the War of 1812. John Quincy Adams was sworn in as the nation's president there and John C. Calhoun had lived and died there. It became a holding facility for temporary prisoners during the Civil War. The U.S. Supreme Court sits in the site today.