Elias Beadle, a resident of Knox County in western Indiana, purchased this lot in July 1845 from Thomas Conner for $900.00 and had this house built shortly thereafter. By the printing of the 1848 New Albany City Directory, Ralph H. and Susan Hurlbut were residing here, although they did not purchase the property until April 1852 for $2,500.00. Mr. Hurlbut was proprietor of his own carpet and furnishing store until January 1849, when he became partners with Mr. Walter Mann in the Hurlbut & Mann carpet and furnishing concern.

The Hurlbuts sold their home in September 1856 to James A. and Marinda Doll for $4,000.00. Mr. Doll had been partners with Ben F. Dodson in the retail dry goods business known as Dodson & Doll, located at the northwest corner of State and Main Streets in downtown New Albany. Later, James would partner with John K. Woodward in the same type of business, but known as Woodward & Doll, and also located at the same downtown location as his previous business concern. James and Marinda lost their property to sheriff sale in June 1870.

By April 1871, Noble C. Butler Esq. and his Mary, had purchased the home, which was next door to his father, John H. Butler, at 504 East Main. The younger Mr. Butler was a lawyer and register in bankruptcy. He was also a US Commissioner. It appears that Noble and Mary moved from New Albany in the late 1870s, but retained ownership of the house through December 1907. During that period of time and through the next owner, Edward T. Slider, the property was used for rental purposes.

In May 1941, Louis A. and Hattie Fischer purchased the structure and it is likely that during this time the house was divided into numerous apartments, as happened with many of the large homes along Main Street.