
From/on the wrong side of the tracks
Class Division
This phrase highlights socioeconomic disparities and is often used to describe societal divisions. 
Idiomatic Geography
The 'tracks' refer to train tracks, which physically divided towns into richer and poorer areas in the past. 
Metaphorical Use
Used metaphorically, it's not about literal geography, but a commentary on class and origins. 