That's actually a major facet of the Confucian class system, which sprang up in China but was adopted across the rest of East Asia as Chinese influence took root many centuries ago. Since merchants were indeed seen as producing nothing by their own hand, but instead move around the goods produced by others, they were ranked lowest in this system, beneath both farmers and craftsmen. (In China the top class were the scholars, thought to be the wisest on account of having to pass the imperial examination; in Japan it was the 'kuge', civil nobility, until they were overthrown & marginalized alongside the imperial family itself by the samurai toward the end of the Heian Period.)