The Governor's Palace was a graphic expression of the crown's presence in the new capital. Its bold outline at the end of Palace Green punctuated Nicholson's town plan and inspired members of Virginia's elite to build the first generation of formal double-pile houses in the Chesapeake. Builder Henry Cary began the residence ten years after the capital's founding, and Lieutenant Governor Alexander Spotswood enriched the building and its gardens from 1710 until about 1720. By mid-century it was considered too small for the new mode of inviting large groups for dancing and eating, and a 75-foot-long wing was added to the rear.

The 1730s copperplate from the Bodleian Library supplied an elevation of the main building and a glimpse of its flankers and courtyard and the parterres immediately behind the main block as they looked before the mid-century changes. Re-creation of the palace buildings relied on this view, the only one that showed the building before its destruction by fire in 1781, as well as on later illustrations of the dependencies; Thomas Jefferson's measured plans of 1779–1781; the so-called Frenchman's Map, of about the same time; and a generous archaeological footprint. Landscape architect Arthur A. Shurcliff had some archaeological and topographic evidence as well, though his garden re-creation is considerably more hypothetical. Together, the palace buildings and gardens represent the largest element of the Williamsburg restoration.