When the U.S. Custom House (WF17) was begun in 1837, it stood at the head of an inlet created by the long wharves that had been built out into the harbor as part of a commercial development that originated in 1803 as designed by Charles Bulfinch. For India Street and India Wharf, Bulfinch's plans called for rows of four-story brick blocks with hipped roofs. None of Bulfinch's documented designs survive, but the building at 25–27 India Street, corner of Milk, is probably one of his original warehouses. Although the storefronts have been altered, the upper stories with marble lintels and band courses resemble the prototype design for the original development that was completed in 1805–1807.

Although sometimes attributed to Bulfinch, Central Wharf, on the opposite side of India Street, is probably not his design. Constructed in 1816–1817, the original Central Wharf comprised a row of fifty-four structures that extended from India Street to where the New England Aquarium (WF15) now stands. Cut in half by the construction of Atlantic Avenue in 1868, Central Wharf was eventually reduced to the existing eight structures. Although altered with additions and changes to windows and doors, Central Wharf is historically important as the last surviving Federal period wharf complex in Boston.