
Not to put too fine a point on it
Directness
The phrase is used to be blunt and doesn't beat around the bush, meaning the speaker gets straight to the point. 
Softening Blow
Despite its directness, it often signals that the speaker wants to soften the impact of a critical statement that follows. 
Colloquial Use
It's a conversational phrase, used more in spoken English or casual writing, not in formal texts. 