This usage often implies a judgement and might be seen as critical or unkind. After the divorce, he let himself go, and his friends were worried.
Pro Tip 2/3
Potential Sensitivity
Be cautious using this phrase as it can touch on personal issues regarding someone’s lifestyle. People noticed she let herself go, but they avoided mentioning it to be polite.
Pro Tip 3/3
Observational Usage
Used when someone else notices a change in a person's self-care. His colleagues remarked that he had let himself go after his injury.