James Mitchell, a New Albany bricklayer, purchased the west half of this lot in 1837 and the remainder in 1846. It is believed that the house was constructed in 1847, with Mr. Mitchell doing the brickwork. There were several other owners before the Honorable Simeon K. Wolfe purchased the home in July 1870 for $9000. It was around this time that the house was enlarged and the structure took on its present Italianate detailing.

Judge Wolfe was born in Georgetown, Indiana in 1824, and began his professional life as a shoemaker. He moved to Corydon as a young adult and began studying law under Judge Porter. He was eventually elected a state senator, in 1851, and also served as editor of the Corydon Democrat newspaper from 1857 to 1865. Wolfe moved to New Albany in 1870, and ten years later was appointed judge of the Circuit Court of Floyd and Clark counties.