Meh. Running by the logic they are going with, this isn't out of character for the actual Samurai class (obviously they wouldn't be hanging out in armor, but still). You couldn't just be a peasant and ignore the presence of the Samurai walking around. They were effectively the elites of a very hierarchical system and you were required to treat them as better than you.