
Be poles apart
Extreme Difference
The phrase emphasizes a very strong or significant difference between two things or people. 
Not Spatial
It's a metaphorical expression; it doesn't mean physical distance, but figurative or ideological distance. 
Negative Connotation
It often implies that the difference is seen as a barrier to agreement or harmony. 