
Be down on your luck
Temporary Hardship
The phrase often implies a hope that one's circumstances will improve soon. 
Empathy Trigger
It's used to express sympathy towards someone facing tough times, not for minor inconveniences. 
Not Derogatory
It doesn't blame the person for their misfortune, but acknowledges their difficult situation. 