Senora Francisca Gallardo was granted a house lot between Bath Street and Vine Street (later renamed Olvera Street) in 1847. In 1881 she gave the adobe she built there to her niece, Eloisa Martinez de Sepúlveda. When Bath Street was widened and made an extension of Main Street in 1886, Eloisa lost 1,600 feet of her mother’s lot and part of the family adobe. As a replacement, the following year she built a combination business and residential building with an unusual Eastlake design. It had a triangular gable and two large bay windows topped with iron cresting. The rough brick façade on Main Street was painted a reddish brown color and penciled with white paint to resemble the precise lines of mortar between the bricks.
The twenty two room building had two large stores fronting on Main Street, and for boarders, fourteen bedrooms and a bathroom on the second floor. Senora Sepúlveda’s private quarters in the rear were separated from the stores by a breezeway. In 1901 she gave the building to her favorite niece and goddaughter Eloisa Martinez de Gibbs who had married Edward Gibbs, a City councilman. Several of the Gibbs children were born in the Sepúlveda House. Senora Sepúlveda died in 1903 and the Gibbs family moved away in 1905, but owned the building until the State of California took it over in 1953.
In the 1930s, after the Mexican marketplace had opened on Olvera Street, Christine Sterling persuaded Forman Brown and his partners to open their “Yale Puppeteers” in the building. She also invited photographers Viroque Baker and Ernest Pratt to set up their studios on the second floor. In the 1940s during World War II, a USO canteen was located in the building, providing a refuge to the thousands of troops passing through Union Station.