Edward Penniman's career as a man of the sea began at age 11. At 21, he was harpooning with the New Bedford, Massachusetts whaling fleet. By the time he was 29, he was master of his own whaling ship. Captain Penniman's seven whaling voyages spanned the globe, taking him to Hawaii and the Arctic. Although the voyages kept him away for years at a time, his wife Augusta and children Eugene, Bessie, and Ned occasionally accompanied him.
In 1868, midway through his successful whaling career, Captain Penniman had this house built for his family. The Pennimans were born and raised here in Eastham, and wherever they roamed, this was home. When constructed, this Second Empire-style house was the most elegant in town, and the family furnished it with treasures from their travels. Upon retiring from the sea at age 53, Captain Penniman remarked. "If I can live 10 years to enjoy my home I shall be happy." He lived a full and healthy life here until his death in 1913 at age 82.