This Queen Anne-style home was built in 1907 on the site of an older (c.1855) structure that was home to the Daniel White family. The new home was purchased for $2000 by the Anthony E. Kremer family. The small cottage next door at 523 East Market is believed to have originally been used as an accessory structure for this home, and was later joined to it.

Anthony Kremer was a well-known New Albany businessman, proprietor of the Opera House news stand at 115 East Spring Street downtown. He would later become owner of a pool hall and news stand on State Street. Tony, who was known for his jovial disposition, had two children - Tony, Jr. and Gertrude, with his wife Catherine.

The property remained in the Kremer family until 1944, when it was purchased by postal carrier Albert Castleain and his wifre, Sarah. The Castleains continued to own the property until 1970.