Fairfield was once the center of the sixteen-hundred-acre plantation of Warner Washington, cousin of George Washington. The Georgian house resembles those of Washington's relatives in eastern Virginia except for its construction in limestone. The two-and-a-half-story hipped-roof house with interior chimneys now has a five-part composition, but the initial construction consisted only of the five-bay central block with its two side wings. The small pedimented front portico is a nineteenth-century addition, and the dormers on the central block, the one-story gable-end additions to the original wings, and the rear additions are twentieth century. By the mid-1780s, Washington possessed 135 slaves, making him the largest slaveholder in what was then Frederick County. His large, stately house was a clear manifestation of this wealth and status. Several other Washington family houses were built in the area, most of which are now located in neighboring Jefferson County, West Virginia.