To the left of the courthouse, the brick former jail has design origins as murky as the crimes of some of its past occupants. Commissioners were appointed in 1823 to contract for the building of the jail. The same year, the second Nelson County Jail (NE2) was built according to Thomas Jefferson's specifications for the Cumberland County Jail. The one-story Nelson County Jail clearly follows the plan submitted by Jefferson, but the two-story three-bay Cumberland County Jail, probably constructed by one of Jefferson's builders, Dabney Cosby, does not. The Nelson jail only housed prisoners, but Cumberland's two-story jail was apparently designed to house the jailor and his office on the first floor. It could be that this is a later jail. The well-built structure continued to serve as the county's jail until 1949. Remodeled the following year for the health department, the building was painted white to give it a more clinical appearance.