This stylish three-story, dark orange brick Greek Revival house, built by ironmaster Greenberry Dorsey, was originally in the heart of town, but soon after its construction, the commercial focus moved a block west beyond 4th Street. It has columned porches on both the north and west elevations. From the 1870s and for over seventy years, the Speer family, which included a U.S. congressman, lived here, but since 1947, the house has been used for office space. The Orbison house (1815) diagonally across the street, similar in size and massing, set the precedent for this large city house.

An eighteen-foot-high slender stone obelisk of 1896 commemorating the Ona Jutta Hage, the Juniata Tribe's standing stone, stands in the 3rd Street parkway, part of a three-blocklong widened street from the railroad tracks to the jail that acts as a town square adjacent to the Speer House.