This structure was built in 1866 along with the Shipman Mansion to the west, by noted architects/builders William and James Banes.  According to local lore, it was used as a children's playhouse, but there is no documentation to prove that story. It was converted into a separate living unit in the early 1940s, when the main house was also divided into apartments. The photo below shows the house in 1971.