Baltimore's first fish market stood near the site of this structure as searly as 1773. The first market building, Centre Market, was authorized by act of the State legislature in 1784. It was also known as Marsh Market since it was built on Thomas Harrison's marsh. Throughout the nineteenth century, market merchants sold dry goods, horses and fresh fruits and getetables. Slave auctions were also held here.
In 1851 what was perhaps America's finest market-type assembly hall was built here between Baltimore and Water Streets to house the Maryland Institute. Presidents Pierce and Fillmore were nominated at national political conventions held here. President Lincoln delivered his famous Liberty Speech here on April 18, 1864, three years after the Pratt Street Riots. The Maryland Institute was destroyed in the Great Baltimore Fire of 1904.
The marketplace was expanded in the rebuilding that followed the Fire. Built in 1907, the current structure served as Baltimore's wholesale Fish Market. It was one of three market buildings erected in a row, all designed by architects Otto Simonson and Theodore Wells Pietsch. The others housed Baltimore's retail and wholesale produce markets. The Fish Market thrived for almost a century before it closed in 1984 when Maryland Wholesale Seafood Center opened in Jessup, Maryland.
In 1997 the Mayor and City Council of Baltimore granted the Baltimore Children's Museum a 100-year lease on the Fish Market, to establish one of the nation's largest children's museums, Port Discovery.