The Marine Observation Tower, also known as the Pagoda, was a gift to the Commonwealth of Virginia and the City of Norfolk from the Taiwan Provincial Government, Republic of China, as a result of a Sister State relationship established in 1981. Governor Dr. Lee Teng-Hui of Taiwan, later President of the Republic of China, proposed the riverfront gift during a good will trade mission to Norfolk in 1983. The tower was built on and around pillars that previously supported a 500,000 gallon molasses storage tank built by Southgate Molasses Company in 1918. The surrounding waterfront industrial area once included cargo shipping piers, warehouses, and rails for C&O Railroad cars barged to the site from Newport News.
All materials for the tower were manufactured in Taiwan and shipped to Norfolk for assembly. The tower was dedicated on May 12, 1989. The Friends of the Pagoda & Oriental Garden Foundation and the City of Norfolk developed the present Garden, featuring plants of Asian origin, granite lanterns and a large Koi pond, as an appropriate setting for the Pagoda. The Garden was dedicated in October 2000.