This brick villa for Massachusetts governor and U.S. senator Christopher Gore replaced a 1793 country seat, which burned in 1799 while Gore served as a diplomat in London. Only a timber-framed, hipped-roofed Federal carriage house (1793) with an oculus over an arched central doorway survives from the earlier farm. The finely detailed brickwork of the new house parallels the best of Boston's urban dwellings, but its full five-part plan reflects the interests of Rebecca Payne Gore. Governor Gore's interest in agricultural reform informed much of the estate's landscaping in the English manner, but Rebecca Gore seems primarily to have designed the house. During visits in 1801–1802 to Paris, she enjoyed “designing houses” with noted French architect Jacques G. Legrand. For many years she corresponded with her husband's friend Senator Rufus King of New York concerning interior design.

An early New England example of a five-part villa plan, Gore Place employs hyphens and side wings, all ornamented with halfround windows, extending from the central block. Arched doorways on either side of three full-length windows light the front dining room. The rear projecting oval wall and tripartite windows of the oval parlor overlook the lawn toward the Charles River. Inside the eastern front door the spiral staircase, a design change during construction, leads to chamber suites on the upper floor and to service spaces above the east wing. Robert Roberts, the Gores' African American butler from 1810 to 1827, published The House Servant's Directory (Boston, 1827), making the house a significant site of domestic service in American labor history. After the Gores' deaths, the estate was acquired by the Lyman family, whose English gardener, Robert Murray, improved its formal landscape. A wood framed farmer's cottage (c. 1835) was added during the Lyman's tenure.