This is one of Virginia's rare surviving examples of rammed earth, or pise, construction. Now a restaurant and gift shop, the house was originally part of Pea Hill, one of the four Brunswick County plantations belonging to the Cocke family of Bremo in Fluvanna County and Belmead (PO15) in Powhatan. At Bremo, General John Hartwell Cocke, apparently in collaboration with his friend Thomas Jefferson, built many pise slave houses. It seems that all of Cocke's Brunswick plantations had rammed earth houses for the slaves but few, if any, of these structures have survived.

To build pise structures, damp soil is packed into wooden forms and dried until it is hard-set. The forms are removed and a thick coat of plaster is applied to the exterior, which is then whitewashed or painted. The building is further protected from the weather with a deep roof overhang. Thus heavily insulated, the structure is relatively cool in summer and warm in winter.

The one-and-a-half-story, three-bay former Overseer's House, a center-passage, single-pile building, is set on plastered fieldstone foundations and has interior-end fieldstone chimneys. The rear wing on the left is much later, and the gable roof's single dormer is probably not original. Only a photograph remains of “The Row,” a lane behind the Overseer's House once lined on each side with nine or ten slave houses. A central chimney separated the two sides of the pise houses, with each side providing housing for one family in one room with a loft above.