Rough up
Definition 1 of 2
Pro Tip 1/3Figurative Usage
Used metaphorically for non-physical intimidation as well. 
The interviewers roughed me up with their tough questions.
Pro Tip 2/3Negative Connotation
Implies a harmful intention and can suggest illegality or immorality. 
The thugs threatened to rough up anyone who refused to pay.
Pro Tip 3/3Physical Confrontation
Generally indicates some level of violence but not severe injury. 
After the match, the losing team tried to rough up the winners.