The view from this point of land between the Appomattox and James rivers is overwhelming. Francis Epees received the property in 1635, and it remained in the family for over 300 years, until sold to the National Park Service in 1979. The house is a large, rambling, U-shaped structure. A small entry porch with Tuscan Doric columns is at the front, and large porches wrap around other sides. The original portion of the house is the central one-and-one-half-story wood-frame section with a traditional doublepile plan. An east wing in the Downing mode which contained another parlor and a library was added c. 1840. In 1850 another addition was built, this time to the west. In the early twentieth century more changes were made. Two rooms restored to c. 1850 can be toured. On the grounds of the main house are several nineteenth-century frame outbuildings: two smokehouses, a dairy, and a kitchen–wash house.

Between June 1864 and April 1865 General U. S. Grant had his tent and later his cabin on the east grounds of the manor. Lincoln visited and used Appomattox Manor as an office. Grant's headquarters cabin, constructed in November 1864, still stands to the east of the house but is mostly rebuilt. Shown at the 1876 Philadelphia Centennial, it remained in Fairmont Park until 1981, when the National Park Service returned it to the site. The vertical timber construction was a common Civil War framing method.