The term often implies that someone is overly strict and doesn't tolerate different behaviors or beliefs. Don’t invite him to the party; he's a puritan and will just spoil the fun.
Pro Tip 2/3
Cultural Association
In modern usage, 'puritan' can have negative connotations, suggesting someone is narrow-minded. Calling her a puritan for her views on TV censorship was quite judgmental.
Pro Tip 3/3
Not Just Religious
While it has religious origins, 'puritan' is used broadly for strict morality in any area. He is a puritan about diet, only eating organic, unprocessed food.