Walter Mann, Esq. and his wife, Ann, purchased this lot from Scott Moore and his wife in December 1854. The Manns were still living on East Main Street as late as 1860, but by the time the New Albany City Directory of 1863 was printed, their new house on East Spring Street had been constructed and they were residing here.

Walter was the cashier for the Bank of the State of Indiana, located at the corner of East Main and Bank streets in downtown New Albany. He was also involved with other New Albany banking interests with such influential citizens as Jesse J. Brown and William S. Culbertson. Mr. Mann moved to Minnesota for health reasons, and auctioned his house in June 1872.

The house was puchased and owned by Ann Reinhard before being sold in April 1900 to John and Fannie Baylor. Mr. Baylor served several terms as Floyd County Treasurer and was on the board of directors of the New Albany Trust Co. He and his wife also had extensive land holdings in New Albany. After John Baxter's unexpected death, widow Fannie sold the property in May 1920 to Dr. James and Maude Baxter.

Their son, Dr. Samuel Baxter, set up his ear, eye, nose and throat practice here, extensively modifying the interior to accomodate its new use. Sold to neighbors Dr. Daniel and Marilyn Cannon in 1958, the building remained in use as a doctor's office until about 2009. It has subsequently been re-remodeled to return its configuration to a single family home.