The phrase often suggests a display of emotion that's intense and possibly more than necessary. Even the judge noticed how he beat his breast about the mistake.
Pro Tip 2/3
Seeking Attention
Using 'beat your breast' can imply that someone is acting in a way to make others notice their distress. She beat her breast in front of the crowd, seeking everyone's eyes.
Pro Tip 3/3
Public Display
This phrase is used for emotions shown publicly, not in private. At the meeting, he began to beat his breast over the recent failure.