An extraordinary house for its time and place, Prospect Hill remains the most imposing mansion in the neighborhood. The three-bay main block, in the Federal style, is tall and square, with a side-hall plan, evidenced by the offcenter entry porch. A long, four-bay side wing was added shortly after the main block was built. The dramatically tall chimneys on the wing are twentieth-century refinements, heightened to improve their drafts.

Prospect Hill is handsomely sited on a hillside and is surrounded by important early outbuildings. Three brick structures—including a privy with six seats—are contemporary with the house. A stone building thought to be the original dwelling may date from the mid-eighteenth century. Several auxiliary buildings have been refitted as part of a bed-and-breakfast.

William Wilson, a merchant who is said to have operated his first store from Prospect Hill, expanded his operations to form a chain of stores between Gerrardstown and Baltimore. He insured his property with the Mutual Assurance Society of Virginia in 1803, and the wording of the policy proves the house has changed little from that time.