According to letters Lovecraft was writing at the time, the whole thing was apparently motivated by the fact that, back in the late 1910s/early 1920s, those rags-to-riches stories were all the rage, while his tales of monsters that make you insane when you look at them weren't, so this was his minor revenge. Also, maybe, just maybe, old H.P. took a look at his last name one day and decided he could make a fortune crafting romantic tales, but it didn't quite work out that way.
In fact, such was Lovecraft's reputation and the literary typecasting that the first time "Sweet Emengarde" was published for public consumption was in the horror collection Beyond the Wall of Sleep, between tales with names like "Herbert West -- Reanimator" and "The Crawling Chaos." Shockingly, its potential audience of bored housewives never managed to find it there.