These lots at the edge of the hill overlooking the river were the site of prominent early residences in the neighborhood. Richard Adams, who owned much of the hill, built his house here around 1790. About twenty years later, William Palmer built a house nearby, which was sold to William Taylor in 1859. Taylor added a second floor, a cupola, and double porches overlooking the river, creating a much grander house. In 1866 the houses were purchased by the Catholic Diocese of Richmond, and the Order of the Visitation of Mary opened a school named Monte Maria in the Adams House. The Romanesque Revival chapel was erected east of the Palmer-Taylor House in 1894–1895. After the nuns received a large donation, they closed the school in 1927 to devote themselves to prayer. The next year the Adams House was demolished to make way for a new dormitory. The nuns maintained a cloistered convent on the site until 1987, when they moved to a smaller, more isolated location. The complex was purchased by a religious organization founded to preserve the site as a place of prayer and worship in the city.