A millstone around/round your neck
Definition 1 of 1
Pro Tip 1/3Negative Connotation
This phrase suggests a severe, stressful burden, often with a sense of being trapped or constrained. 
His past mistakes have become a millstone around his neck, keeping him from moving forward.
Pro Tip 2/3Metaphorical Use
Used metaphorically to describe non-physical problems or responsibilities that are very heavy and difficult to bear. 
The failing family business became a millstone round her neck, consuming all her time and energy.
Pro Tip 3/3Use in Serious Context
It’s typically used in serious matters, not for trivial inconveniences. 
The legal dispute with the municipality has turned into a millstone around their necks.