Galveston's most famous architectural landmark is the picturesque granite and limestone house that Clayton designed for lawyer, state legislator, and one-term U.S. congressman Walter Gresham, who came to Galveston in 1866 from his native Virginia as a Confederate veteran. An intrepid promoter of transportation improvements, Gresham was one of the original investors in the Gulf, Colorado and Santa Fe Railway. It seems to have been the sale of the railroad in 1886 that gave him the resources to build a house of this scale and material splendor. Clayton made the Gresham house extraordinary by piling three floors atop a high raised basement, facing them with polygonal towered bays and cylindrical turrets and chimney stacks.

In 1923, Gresham's heirs sold the house to the Catholic Diocese of Galveston as a residence for Bishop Christopher Byrne, who occupied it until his death in 1950. In 1963, four years after the diocesan see was moved to Houston, the Diocese of Galveston-Houston opened the Gresham House as a historic house museum, the Bishop's Palace, the first of the grand Broadway houses to become publicly accessible. Although still owned by the Archdiocese of Galveston-Houston, the Bishop's Palace is now operated by the Galveston Historical Foundation.