Yes, though the argument is not limited to just usurers. Any middleman, including shopkeepers, bred similar kind of hatred. That is the argument.

Of course, in Europe it was combined with a dislike for the Jewish religion, which was a toxic brew. But it's worth noting - as Sowell, IIRC does not make in his book - that there was not that much anti-Judaism in the Early Middle Ages. Only from the late 11th century did you get it, and that happens to be the start of the 'commercial' revolution.