
It isn’t my, his, etc. thing
Colloquial Usage
The phrase is casual and common in everyday conversation among friends or peers. 
Subtle Refusal
It's a polite way to decline something without offending someone by directly criticizing it. 
Personal Preference
The phrase emphasizes personal taste and individual interest, or lack thereof. 