A strange fusion of colonial and late-nineteenth-century styles, Milford's most famous building was home to Reverend Sydenham Thorne, co-founder of the town. It was once thought that he built the house c. 1785, but, in fact, it is considerably older, consisting of a center passage Georgian structure added to an existing frame wing (at rear; with brick nogging). Wooden lintels imitate stone, as often seen in Kent County. In the 1870s, Colonel Henry B. Fiddeman, banker and president of railroad and steamship companies, added sharp gables to the main block and wings and heightened the roofline. “Rebuilt the house frame,” a carpenter wrote on a board inside. Fortunately, the interior was not touched and retains its colonial paneling. Tenant farmers lived here in the early twentieth century, when the Draper family owned it. Milford Historical Society, founded by Catherine Downing and others, took over in 1962 and has restored it several times, initially under Albert Kruse (1963), who removed the Fiddeman front porch, renewed the windows with colonial-type sash, and found bits of original weatherboarding under imitation brick siding on the rear wing. Fortunately, funds were lacking for removal of the weird late-nineteenth-century accretions of gables and trim at the roofline that made the house, for an unsympathetic Harold Eberlein (1962), “a monument to insanity.”