Free online at Archive.org, the book Providence in Colonial Times (1912). Not listed in Joshi’s Lovecraft’s Library, but in H.P. Lovecraft: A Life (p.418) he states…

Lovecraft began reading Providence in Colonial Times at the very end of July 1925. Since he could not check the book out of the New York Public Library [he] had to read it in the genealogical reading room during library hours”.