The Miles Brewton House, with surviving eighteenth-century dependencies including a kitchen, laundry, and carriage house, is one of the most complete Georgian town house complexes in the United States. The double-pile brick house is two stories high over a raised basement. A classical, double portico with Doric columns on the first level and Ionic columns on the second level dominates the east facade. The dentilled pediment contains an elliptical window. The east facade is five bays wide, with a grand Palladian-style door. Brick jack arches top the six-over-six sash windows.

The first floor has a central hall with mahogany doors and a mahogany stair. Each of the principal rooms has a marble mantel. The second floor dining room contains intricately carved woodwork that is likely the work of London-born cabinetmakers John Lord and Thomas Woodin, who would have had ready access to popular pattern books of the period that may have inspired the design of the house and its details.

The house has been altered a number of times, beginning in the 1820s with the addition of red sandstone sills and the altering of the window sashes. At that time, a brick wall was built around the perimeter of the property; this wall was fortified after the failed Denmark Vesey slave insurrection of 1822. In the 1840s, several Greek Revival additions were made to the west (rear) elevation that enclosed the once projecting stair tower and the carriage house was altered with a Gothic Revival facade. During the same decade, the property was subdivided, creating separate lots on Legare Street.

Built for Miles Brewton, in the early twentieth century the house served as the residence of Susan Pringle Frost. Frost played a decisive role in presevering this and many other nearby historic buildings through her business as a real estate broker. In 1920 she led the formation of the Society for the Preservation of Old Dwellings, which grew into the Preservation Society of Charleston, an organization whose activities have not only assured the preservation of much of central Charleston but, even more importantly, have inspired preservation movements all across America. Although the Miles Brewton House was severely damaged in an 1886 earthquake and 1938 tornado, it has been immaculately restored and remains privately owned.