The architects of the Frederick Vanderbilt House chose its rustic-hewn gray-brown granite as fitting for this most dramatic of Newport sites, where a southern point of the island confronts the Atlantic waters with an eroded, rocky coastline. Built for a Vanderbilt who was as much concerned for his privacy as some of his neighbors were for public display, the house is all but screened from public view, by streetside walls, a planting design (originally by the Olmsted firm but now altered), and the coarse cliffs that rise from the sea.

Borrowing composition and materials from English manor house sources, the house is low and horizontal, its spaces divided on the exterior by several cross gables and the cant of its service wing. The small entryway leads to a two-story English-style, oak-paneled hall, off which the other rooms of the house are arranged. The heavy ceiling beams and the oversized fireplace fitted with gargantuan fire dogs underscore the closed, medieval character of the Peabody and Stearns design. Later additions and interior renovations by Horace Trumbauer, such as the solarium overlooking the ocean, are more Gallic in their open classicism, spatial proportions, arched windows, and use of color, reflecting the shift in domestic taste encouraged by Edith Wharton, Ogden Codman, and others. Today the house holds an eclectic, idiosyncratic array of art, furniture, and even clothing once owned by Doris Duke. In 1925, as a thirteen-year-old, she inherited the house from her father, the energy and tobacco magnate who established Duke University and had purchased the property a few years earlier. Several years after her death in 1993, it was opened to the public as a historic house museum.