This expression is not about physical location, but about being in a different life situation. Sometimes I feel I'm on the other side of the fence, with all my friends married.
Pro Tip 2/3
Grass Is Greener
Implies that a situation may seem more desirable from a distance, which may not be the case up close. Looking at celebrities, it's easy to think they live on the other side of the fence.
Pro Tip 3/3
Cultural Perception
Can reflect the idea that different cultures or lifestyles are envied or idealized. Travelers often find themselves imagining life on the other side of the fence.