When society expects men to do things men are increasingly asking "What's in it for us?". The answer is often nothing at best and something very painful (socially, physically, or financially) at worst. There's nothing to be gained in intervening in such a situation. You could however get beaten, stabbed, or shot. You'll also get called a racist and get charged with hate crimes if the attackers are non-white as they often are.