An M.A. thesis from a religious university, currently only available via Google “Quick View”…

Geoffrey Reiter (2005), “A Dark Poem”: Lovecraft and his Puritans.

“This work examines the fiction and letters of H.P. Lovecraft […] and seeks to explore the way in which he incorporated Puritanism into his works and artistic philosophy, despite the fact that his materialistic worldview would not permit him to accept Puritan doctrinal tenets.”

It can be found with a Google “Quick View” link, via the following Google Search query:

   lovecraft “Puritanism and Christianity” filetype:pdf