At 66 College Street, Lovecraft could look into the stacks of the John Hay Library from his kitchen window. There is another building there now, but if you visit the place you'll see how very close it was.
I can recall off the top of my head that he used the Public Library in NYC when reading Gertrude Selwyn Kimball's book on colonial Providence, but that particular book he had to read there. And he mentions in a letter that he met Arthur Fredlund at the Public Library in Providence. I don't know whether the various libraries of Brown University were open to the public or just to academics.