Jews could charge interest on loans to non-Jews but not to one another (defaulting on a loan to another Jew was very strictly forbidden) and didn't necessarily have to tithe to a ruler, as well as being free of compelled donations to Rome/Constantinople/Mecca.

I know the history, but the existence of their wealth in the first place is the puzzling part.

As for the second point there were probably no shortage of mercenaries with various levels of morality willing to take "repossession" jobs.

Could be! I've often wondered if "blood libel" didn't originate with methods of compelling repayment.