This use often implies playful or mischievous intent rather than a desire to truly upset someone. He'd make silly jokes just to get a rise out of her.
Pro Tip 2/3
Informal Contexts
Typically used in informal settings among friends or family, not in professional environments. At family gatherings, Uncle Roy tries to get a rise out of everyone with his pranks.
Pro Tip 3/3
Degree of Annoyance
The reaction sought is usually mild annoyance, so the phrase is not used for serious provocation. My brother would always eat with his mouth open to get a rise out of me.