Belden Lane on Puritan sexuality:

Puritans from Francis Rous and Richard Sibbes in England to John Cotton and Edward Taylor in New England rang all the changes on the erotic language of the Song of Songs. They described spiritual devotion as a matter of “lusting” after Christ as bridegroom. They pictured the heart as a marriage bed prepared for the divine lover. The language of foreplay and orgasm became as common in Puritan preaching as anything found in the bridal mysticism of Bernard of Clairvaux or Mechthild of Magdeburg and Teresa of Avila.

All this makes it clear, then, why the Puritans had to be so careful in cautioning themselves about the danger of sexual sins. It wasn’t that they were sexually repressed, straight-laced prudes—eager to put a bright red “A” on the dress of every Hester Prynne in the colonies. They simply had a spirituality which fostered so much interior passion that setting appropriate boundaries for their exterior behavior was absolutely necessary.

 -- Belden C. Lane, Ravished by Beauty: The Surprising Legacy of Reformed Spirituality, 19

Probably because I've spent so much time reading their works, I'm always a bit surprised when I encounter attitudes toward the Puritans which dismiss them as mindless prudes (immortalized by H.L. Mencken's definition: "Puritanism: The haunting fear that someone, somewhere, may be happy"). In reading travel guides about Boston, reading interpretive plaques, and even in talking to other researchers at a historical archive I'm consistently surprised by the way Puritan scholars used to paint, and popular understanding continues to paint, Puritan sensibilities. I think we tend to look back through the lens of Victorianism and confuse their reading of history with actual history. The Victorians saw a carefulness which they confused with their own prudishness, stripped that carefulness of its sexuality, and held up the Puritans as their prudish ideal. Belden Lane has done an excellent job here of overcoming some of the missconception.