If all you can point to is a handful of cases of some serious side effect or complication, that's like 0.000001%, which isn't relevant at all.

As others have alluded to, it's insignificant so far. Vaccines are normally tested for a sufficient length of time to have a good picture of the long-term effects of the vaccine, something conspicuously lacking from this vaccine.

While I've had the vaccine myself (I'm recommended to get the 'flu vaccine annually anyway, and I'm extremely unlikely to fall pregnant) I completely understand somebody in a different situation making different choices here.